Inter-operative switching of tools in a robotic surgical system

ABSTRACT

A system includes manipulators and a controller. The controller is configured to detect mounting of an imaging device to a first manipulator of the manipulators, determine a first reference frame for the imaging device based on the mounting of the imaging device to the first manipulator, control a tool relative to the first reference frame by controlling a relative position and orientation of a tip of the tool relative to the imaging device in the first reference frame by correlating movement of a master input control to movement of the tool in the first reference frame, detect mounting of the imaging device to a second manipulator of the manipulators, the second manipulator being different from the first manipulator, determine a second reference frame for the imaging device based on the mounting of the imaging device to the second manipulator, and control the tool relative to the second reference frame.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/677,964, filed Aug. 15, 2017, which is a continuation ofU.S. patent application Ser. No. 15/298,130, filed Oct. 19, 2016, whichis a continuation of U.S. patent application Ser. No. 14/218,300, filedMar. 18, 2014, and claims priority to U.S. Provisional PatentApplication No. 61/793,227 filed Mar. 15, 2013. The disclosures of whichare incorporated herein by reference in their entireties.

The present application is generally related to the followingcommonly-owned applications: U.S. Provisional Patent Application No.61/683,495, filed Aug. 15, 2012, entitled “Phantom Degrees of Freedomfor Manipulating the Movement of Robotic Systems”, U.S. ProvisionalPatent Application No. 61/654,764, filed Jun. 1, 2012, entitled“Commanded Reconfiguration of a Surgical Manipulator Using the NullSpace”, U.S. patent application Ser. No. 12/494,695, filed Jun. 30, 2009(now U.S. Pat. No. 8,768,516), entitled “Control of Medical RoboticSystem Manipulator About Kinematic Singularities;” U.S. patentapplication Ser. No. 12/406,004, filed Mar. 17, 2009 (now U.S. Pat. No.8,271,130), entitled “Master Controller Having Redundant Degrees ofFreedom and Added Forces to Create Internal Motion;” U.S. patentapplication Ser. No. 11/133,423, filed May 19, 2005 (now U.S. Pat. No.8,004,229), entitled “Software Center and Highly Configurable RoboticSystems for Surgery and Other Uses;” U.S. patent application Ser. No.10/957,077, filed Sep. 30, 2004 (now U.S. Pat. No. 7,594,912), entitled“Offset Remote Center Manipulator For Robotic Surgery;” and U.S.application Ser. No. 09/398,507, filed Sep., 17, 1999 (now U.S. Pat. No.6,714,839), entitled “Master Having Redundant Degrees of Freedom,” thedisclosures of which are incorporated herein by reference in theirentireties.

BACKGROUND

The present invention generally provides improved surgical and/orrobotic devices, systems, and methods.

Minimally invasive medical techniques are aimed at reducing the amountof extraneous tissue which is damaged during diagnostic or surgicalprocedures, thereby reducing patient recovery time, discomfort, anddeleterious side effects. Millions of surgeries are performed each yearin the United States. Many of these surgeries can potentially beperformed in a minimally invasive manner. However, only a relativelysmall number of surgeries currently use these techniques due tolimitations in minimally invasive surgical instruments and techniquesand the additional surgical training required to master them.

Minimally invasive telesurgical systems for use in surgery are beingdeveloped to increase a surgeon's dexterity as well as to allow asurgeon to operate on a patient from a remote location. Telesurgery is ageneral term for surgical systems where the surgeon uses some form ofremote control, e.g., a servomechanism, or the like, to manipulatesurgical instrument movements rather than directly holding and movingthe instruments by hand. In such a telesurgery system, the surgeon isprovided with an image of the surgical site at the remote location.While viewing typically a three-dimensional image of the surgical siteon a suitable viewer or display, the surgeon performs the surgicalprocedures on the patient by manipulating master control input devices,which in turn control the motion of robotic instruments. The roboticsurgical instruments can be inserted through small, minimally invasivesurgical apertures to treat tissues at surgical sites within thepatient, such apertures resulting in the trauma typically associatedwith open surgery. These robotic systems can move the working ends ofthe surgical instruments with sufficient dexterity to perform quiteintricate surgical tasks, often by pivoting shafts of the instruments atthe minimally invasive aperture, sliding of the shaft axially throughthe aperture, rotating of the shaft within the aperture, and/or thelike.

The servomechanism used for telesurgery will often accept input from twomaster controllers (one for each of the surgeon's hands) and may includetwo or more robotic arms or manipulators. Mapping of the hand movementsto the image of the robotic instruments displayed by the image capturedevice can help provide the surgeon with accurate control over theinstruments associated with each hand. In many surgical robotic systems,one or more additional robotic manipulator arms are included for movingan endoscope or other image capture device, additional surgicalinstruments, or the like.

A variety of structural arrangements can be used to support the surgicalinstrument at the surgical site during robotic surgery. The drivenlinkage or “slave” is often called a robotic surgical manipulator, andexemplary linkage arrangements for use as a robotic surgical manipulatorduring minimally invasive robotic surgery are described in U.S.Provisional Patent Application No. 61/654,764 filed Jun. 1, 2012,entitled “Commanded Reconfiguration of a Surgical Manipulator Using theNull Space”, and U.S. Pat. Nos. 6,758,843; 6,246,200; and 5,800,423, thefull disclosures of which are incorporated herein by reference in theirentirety. These linkages often make use of a parallelogram arrangementto hold an instrument having a shaft. Such a manipulator structure canconstrain movement of the instrument so that the instrument shaft pivotsabout a remote center of spherical rotation positioned in space alongthe length of the rigid shaft. By aligning this center of rotation withthe incision point to the internal surgical site (for example, with atrocar or cannula at an abdominal wall during laparoscopic surgery), anend effector of the surgical instrument can be positioned safely bymoving the proximal end of the shaft using the manipulator linkagewithout imposing dangerous forces against the abdominal wall.Alternative manipulator structures are described, for example, in U.S.Pat. Nos. 7,594,912, 6,702,805; 6,676,669; 5,855,583; 5,808,665;5,445,166; and 5,184,601, the full disclosures of which are incorporatedherein by reference in their entirety.

While the new robotic surgical systems and devices have proven highlyeffective and advantageous, still further improvements would bedesirable. In some cases, it is desirable to change portions or all ofmanipulator assemblies, where a manipulator assembly may include a tool(e.g., a surgical tool) connected to a manipulator (e.g., robotic arm).For example, it may be desirable to change a robotic arm from aparallelogram arrangement that structurally constrains movement about aremote center to an alternative manipulator structure that uses, e.g.,software control to constrain movement about the remote center. Foranother example, it may be desirable to change a tool connected to amanipulator from, e.g., one with clamping jaws to one with an endoscope.

In any event, a different manipulator assembly will often have differentcharacteristics, such as a different number of degrees of freedom,different types of degrees of freedom, etc. Accordingly, the samecontroller for controlling the different manipulator assemblies cannotbe used, but rather a different controller that performs, e.g.,calculations in joint space, must be used that is customized to eachspecific tool and/or manipulator. The use of different controllersresults in added layers of complexity that make the system more prone toerror, and may effectively limit the use of new manipulators and/ortools with a preexisting system. While some techniques for providingsystem compatibility with new tools have been disclosed, such as thosediscussed in U.S. patent application Ser. No. 12/114,082 filed May 2,2008 (now U.S. Pat. No. 7,983,793), entitled “Tool Memory-Based SoftwareUpgrades for Robotic Surgery,” the disclosure of which is incorporatedherein by reference in its entirety, further improvements are stilldesired.

For these and other reasons, it would be advantageous to provideimproved devices, systems, and methods for surgery, robotic surgery, andother robotic applications. It would be particularly beneficial if theseimproved technologies provided the ability to conveniently switchbetween different types of manipulators and/or tools in an error-freefashion while keeping system complexity and costs low.

BRIEF SUMMARY

The present invention generally provides improved robotic and/orsurgical devices, systems, and methods. In one embodiment, a method forcontrolling a telesurgical system is disclosed. The method includesvarious operations, including controlling a first tool connected to afirst manipulator of the system, and a second tool connected to a secondmanipulator of the system. The method further includes detecting a swapof the tools such that the first tool is connected to the secondmanipulator and the second tool is connected to the first manipulator.The method also includes controlling the first tool connected to thesecond manipulator and the second tool connected to the firstmanipulator.

In accordance with another embodiment, another method for controlling atelesurgical system is disclosed. The method includes variousoperations, including determining whether a tool is connected to amanipulator of the system, acquiring a mapping for the tool when it isdetermined that the tool is connected to the manipulator, controllingthe tool using the acquired mapping, determining whether the tool isremoved from the manipulator and a new tool is connected to themanipulator, acquiring a new mapping for the new tool when it isdetermined that the tool is removed from the manipulator and a new toolis connected to the manipulator; and controlling the new tool using theacquired new mapping.

In accordance with yet another embodiment, a telesurgical system forperforming minimally invasive surgery through an aperture of a patientis disclosed. The system includes a plurality of robotic manipulatorseach operable to receive one of a plurality of tools including animaging device and a surgical instrument, and a controller. Thecontroller may be operable to perform a variety of functions. Forexample, the controller may control an imaging device connected to afirst manipulator of the robotic manipulators and a surgical instrumentconnected to a second manipulator of the robotic manipulators, detect aswap of the imaging device and the surgical instrument such that theimaging device is connected to the second manipulator and the surgicalinstrument is connected to the first manipulator, and control theimaging device connected to the second manipulator and the surgicalinstrument connected to the first manipulator.

For a fuller understanding of the nature and advantages of the presentinvention, reference should be made to the ensuing detailed descriptionand accompanying drawings. Other aspects, objects and advantages of theinvention will be apparent from the drawings and detailed descriptionthat follows.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is an overhead view of a robotic surgical system in accordancewith embodiments of the present invention.

FIG. 1B diagrammatically illustrates the robotic surgical system of FIG.1A.

FIG. 2 is a perspective view of the surgeon console of FIG. 1A.

FIG. 3 is a perspective view of the electronics cart of FIG. 1A.

FIG. 4 is a perspective view of a patient side cart having a pluralityof manipulator arms each supporting a surgical instrument.

FIG. 5 is a perspective view of a manipulator arm in accordance with anembodiment.

FIG. 6A is a perspective view of a robotic surgery tool that includes anend effector having opposing clamping jaws in accordance with anembodiment.

FIG. 6B illustrates a wristed endoscope in accordance with anembodiment.

FIG. 6C is a perspective view of the distal end of an overtube withsuction ports in accordance with an embodiment.

FIG. 6D illustrates a non-wristed endoscope in accordance with anembodiment.

FIG. 7A is a perspective view of a master control input device inaccordance with an embodiment.

FIG. 7B is a perspective view of a gimbal or wrist of the input deviceof FIG. 7A.

FIG. 7C is a perspective view of an articulated arm of the input deviceof FIG. 7A.

FIG. 8A is part of a robotic system including a manipulator assembly anda support structure according to a first embodiment.

FIG. 8B is part of a robotic system including a manipulator assembly anda support structure according to a second embodiment.

FIG. 8C is part of a robotic system including a manipulator assembly anda support structure according to a third embodiment.

FIG. 9A illustrates a mapping between connector input/output elementsand joint space interface elements according to a first embodiment.

FIG. 9B illustrates a mapping between connector input/output elementsand joint space interface elements according to a second embodiment.

FIG. 9C illustrates a mapping between joint space interface elements andwork space interface elements according to an embodiment.

FIG. 10 illustrates mappings between connector input/output elements,joint space interface elements, and work space interface elementsaccording to an embodiment.

FIG. 11A shows a first set of groupings of the degrees of freedom of afirst manipulator assembly being controlled by the joint space interfaceelements.

FIG. 11B shows a second set of groupings of the degrees of freedom of asecond manipulator assembly being controlled by the joint spaceinterface elements.

FIG. 12A shows a connector/joint space map according to an embodiment.

FIG. 12B shows a joint space/work space map according to an embodiment.

FIG. 13A shows a patient side cart including an imaging device coupledto a first manipulator arm and a surgical tool coupled to a secondmanipulator arm.

FIG. 13B shows a patient side cart including an imaging device coupledto the second manipulator arm and a surgical tool coupled to the firstmanipulator arm.

FIG. 14 shows a sequence of operations that may be used to control onefrom a number of possible tools connected to the same manipulator arm.

FIG. 14A shows a sequence of operations that may be used when an imagingdevice is removed from a first manipulator arm and coupled to a secondmanipulator arm so as to maintain correspondence between movement ofsurgical tools imaged by the image capture device as displayed to asystem user and input devices being manipulated by the system user.

FIGS. 14B and 14C show tool tips before and after a tool swap,respectively, and schematically indicate associated changes to a camerareference coordinate system for controlling surgical instruments.

FIG. 15 shows a sequence of operations for acquiring mappings for a toolaccording to an embodiment.

FIG. 16A shows a sequence of operations that may be used to control atool using an acquired mapping in accordance with one embodiment.

FIG. 16B shows a sequence of operations that may be used to control atool using an acquired mapping in accordance with another embodiment.

DETAILED DESCRIPTION

Embodiments of the present invention generally provide improvedtechniques for controlling a number of different manipulator assemblies.Some embodiments are particularly advantageous for use with surgicalrobotic systems in which a plurality of surgical tools or instrumentsare mounted on and moved by an associated plurality of roboticmanipulators during a surgical procedure. The robotic systems will oftencomprise telerobotic, telesurgical, and/or telepresence systems thatinclude processors configured as master-slave controllers. By providingrobotic systems employing processors appropriately configured to controla number of different manipulator assemblies, such as different roboticarms and/or surgical tools, the flexibility of the robotic systems inperforming surgical procedures may be significantly increased.

The robotic manipulator assemblies described herein will often include arobotic manipulator and a tool mounted thereon (the tool oftencomprising a surgical instrument in surgical versions), although theterm “robotic assembly” will also encompass the manipulator without thetool mounted thereon. The term “tool” encompasses both general orindustrial robotic tools and specialized robotic surgical instruments,with these later structures often including an end effector which issuitable for manipulation of tissue, treatment of tissue, imaging oftissue, or the like. The tool/manipulator interface will often be aquick disconnect tool holder or coupling, allowing rapid removal andreplacement of the tool with an alternate tool. The manipulator assemblywill often have a base which is fixed in space during at least a portionof a robotic procedure, and the manipulator assembly may include anumber of degrees of freedom between the base and an end effector of thetool. For example, the manipulator assembly may include kinematicdegrees of freedom of a manipulator as well as kinematic degrees offreedom of a tool connected to the manipulator. The combination of thesemay be referred to herein as “manipulator degrees of freedom”, and aretypically defined in joint space. Actuation of the end effector (such asopening or closing of the jaws of a gripping device, energizing anelectrosurgical paddle, activating air pressure for a vacuum, or thelike) will often be separate from, and in addition to, these manipulatorassembly degrees of freedom. These may be referred to herein as“actuation degrees of freedom”.

The end effector (or, more generally, the control frame, as describedbelow) will typically move in the work space with between two and sixdegrees of freedom, but may move in work spaces with fewer than two orgreater than six degrees of freedom. The degrees of freedom of the endeffector (or, more generally, the degrees of freedom of the controlframe) may be referred to herein as “end effector degrees of freedom”,and are typically defined in a Cartesian work space (described below).As used herein, the term “position” encompasses both location (e.g., x,y, z coordinates) and orientation (e.g., pitch, yaw, roll). Hence, achange in a position of an end effector (for example) may involve atranslation of the end effector from a first location to a secondlocation, a rotation of the end effector from a first orientation to asecond orientation, or a combination of both. When used for minimallyinvasive robotic surgery, movement of the manipulator assembly may becontrolled by one or more processors of the system so that a shaft orintermediate portion of the tool or instrument is constrained to a safemotion through a minimally invasive surgical access site or otheraperture. Such motion may include, for example, axial insertion of theshaft through the aperture site into a surgical work space, rotation ofthe shaft about its axis, and pivotal motion of the shaft about a pivotpoint at the aperture site.

In one particular embodiment, kinematic degrees of freedom of amanipulator assembly may be controlled by driving one or more joints viathe controller using motors of the system, the joints being drivenaccording to coordinated joint movements calculated by a processor ofthe controller. Mathematically, the controller may perform at least someof the calculations of the joint commands using vectors and/or matrices,some of which may have elements corresponding to configurations orvelocities of the joints. The range of alternative joint configurationsavailable to the processor may be conceptualized as a joint space. Thejoint space may, for example, have as many dimensions as the manipulatorassembly has degrees of freedom, and in some exemplary embodiments, thejoint space may have more dimensions than the manipulator assembly hasdegrees of freedom as the manipulator assembly may lack at least onedegree of freedom necessary to fully define the position of an endeffector associated with the manipulator assembly. Further, a particularconfiguration of the manipulator assembly may represent a particularpoint in the joint space, with each coordinate corresponding to a jointstate of an associated joint of the manipulator assembly where anassociated joint of the manipulator exists.

In an exemplary embodiment, the system includes a controller in which acommanded position and velocity of a feature in the work space, denotedhere as its Cartesian space, are inputs. The feature may be any featureon the manipulator assembly or off the manipulator assembly which can beused as a control frame to be articulated using control inputs. Anexample of a feature on the manipulator assembly, used in many examplesdescribed herein, would be the tool-tip. Another example of a feature onthe manipulator assembly would be a physical feature which is not on thetool-tip, but is a part of the manipulator assembly, such as a pin or apainted pattern. An example of a feature of the manipulator assemblywould be a reference point in empty space which is exactly a certaindistance and angle away from the tool-tip. Another example of a featureoff the manipulator assembly would be a target tissue whose positionrelative to the manipulator assembly can be established. In all of thesecases, the end effector is associated with an imaginary control framewhich is to be articulated using control inputs. However, in thefollowing, the “end effector” and the “tool tip” are used synonymously.Although generally there is no closed form relationship which maps adesired Cartesian space end effector position to an equivalent jointspace position, there is generally a closed form relationship betweenthe Cartesian space end effector and joint space velocities. Thekinematic Jacobian is the matrix of partial derivatives of Cartesianspace position elements of the end effector with respect to joint spaceposition elements. In this way, the kinematic Jacobian captures thekinematic relationship between the end effector and the joints of themanipulator assembly. In other words, the kinematic Jacobian capturesthe effect of joint motion on the end effector.

Many (but not all) of the manipulator assemblies described herein havefewer degrees of freedom available for use than those that are typicallyassociated with full control over the positioning of an end effector ina work space (where full control of the end effector requires endeffector degrees of freedom including three independent translations andthree independent orientations). That is, the manipulator assemblies mayhave an insufficient number or type of degrees of freedom forindependently controlling the six end effector degrees of freedom. Forexample, a rigid endoscope tip without an articulating wrist may bemissing two degrees of freedom at the wrist. Accordingly, the endoscopemay have only four degrees of freedom for positioning the end effector,rather than six, thus potentially constraining the motion of theendoscope.

However, some of the manipulator assemblies described herein have agreater number of degrees of freedom than that required to fully controlthe positioning of the end effector (where full control of the endeffector requires end effector degrees of freedom including threeindependent translations and three independent orientations), but due tothe type or arrangement of the joints of the manipulator assemblies, themanipulator assemblies still cannot fully control the positioning of theend effector. For example, a manipulator assembly may have sevenmanipulator degrees of freedom, but three of those are redundant. As aresult, the end effector effectively has five degrees of freedom. Insome embodiments, the manipulator assemblies may have sufficient degreesof freedom to fully control the positioning of an end effector.

Regardless of the number of degrees of freedom available for controllingthe position of the end effector, the manipulator assemblies describedherein may also facilitate additional degrees of freedom for actuating atool (i.e., actuation degrees of freedom). For example, the manipulatorassemblies may be configured to mount a tool having an electrocauteryprobe operable to, e.g., heat select tissue upon activation, whereactivation/deactivation of heat is a degree of freedom. For anotherexample, the manipulator assemblies may be configured to mount a toolhaving a vacuum operable to, e.g., apply suction forces around selecttissue upon activation, where actuating the suction forces is a degreeof freedom. For yet another example, the manipulator assemblies may beconfigured to mount a tool having a grip, where actuation of the grip isa degree of freedom. For even yet another example, the manipulatorassemblies may be configured to mount a tool having a grip and a cutter,where actuation of the grip is a degree of freedom and actuation of thecutter is a degree of freedom. In such cases, these additional degreesof freedom are not kinematic as they do not affect the position (i.e.,location and orientation) of the end effector. Accordingly, theseadditional degrees of freedom may be referred to as ‘non-kinematic’ or‘actuation’ degrees of freedom. This is in contrast to kinematic degreesof freedom (e.g., the manipulator degrees of freedom described herein),as kinematic degrees of freedom impact the position of the end effector.

The term “state” of a joint or the like will often herein refer to thecontrol variables associated with the joint. For example, the state ofan angular joint can refer to the angle defined by that joint within itsrange of motion, and/or to the angular velocity of the joint. Similarly,the state of an axial or prismatic joint may refer to the joint's axialposition, and/or to its axial velocity. While many of the controllersdescribed herein comprise velocity controllers, they often also havesome position control aspects. Alternative embodiments may relyprimarily or entirely on position controllers, acceleration controllers,or the like. Many aspects of control systems that can be used in suchdevices are more fully described in U.S. Pat. No. 6,699,177, the fulldisclosure of which is incorporated herein by reference. Hence, so longas the movements described are based on the associated calculations, thecalculations of movements of the joints and movements of an end effectordescribed herein may be performed using a position control algorithm, avelocity control algorithm, a combination of both, and/or the like.

In many embodiments, the tool of an exemplary manipulator arm pivotsabout a pivot point adjacent a minimally invasive aperture. In someembodiments, the system may utilize a hardware remote center, such asthe remote center kinematics described in U.S. Pat. No. 6,786,896, theentire contents of which are incorporated herein in its entirety. Suchsystems may utilize a double parallelogram linkage which constrains themovement of the linkages such that the shaft of the instrument supportedby the manipulator pivots about a remote center point. Alternativemechanically constrained remote center linkage systems are known and/ormay be developed in the future. In other embodiments, the system mayutilize software to achieve a remote center, such as described in U.S.Pat. No. 8,004,229, the entire contents of which are incorporated hereinby reference. In a system having a software remote center, the processorcalculates movement of the joints so as to pivot an intermediate portionof the instrument shaft about a desired pivot point, as opposed to amechanical constraint. By having the capability to compute softwarepivot points, different modes characterized by the compliance orstiffness of the system can be selectively implemented. Moreparticularly, different system modes over a range of pivotpoints/centers (e.g., moveable pivot points, whereby the softwaredefined pivot point may be moved from one location to another; passivepivot points, whereby a patient's body wall is relied on to enforce theconstraint of going through the ‘center’; fixed/rigid pivot point; softpivot points; etc.) can be implemented as desired.

In many configurations, robotic surgical systems may include mastercontroller(s) having a number of degrees of freedom fewer than, morethan, or equal to the number of degrees of freedom which the remotelycontrolled robotic manipulator arms and/or tools have. In such cases,Jacobian based or other controllers used to control the roboticmanipulator arms and/or tools typically provide complete mathematicalsolutions and satisfactory control. For example, fully controlling theposition (i.e., location and orientation) of a rigid body can employ sixindependently controllable degrees of freedom of the rigid body, whichincludes three degrees of freedom for translations and three degrees offreedom for rotations. This lends itself nicely to a Jacobian basedcontrol algorithm in which a 6×N Jacobian matrix is used.

Although manipulator assemblies having a variety of degrees of freedomare disclosed herein, including assemblies having fewer than, the samenumber as, or more than the six degrees of freedom for fully controllingthe position of an end effector, many embodiments of these assemblieslack at least one degree of freedom for fully controlling the positionof the end effector. While the manipulator assemblies may lack one ofthese degrees of freedom, the input device controlling the manipulatorassembly (e.g., a master control input device) may include the lackingdegree of freedom. In accordance with embodiments of the presentinvention, in response to an input controlling the degree(s) of freedommissing at the manipulator assembly, the other degrees of freedomavailable at the manipulator assembly may provide motions so as tosimulate control of the missing degree(s) of freedom. This may be doneby using a kinematic model of the manipulator assembly that includes andperforms calculations for the missing manipulator degree(s) of freedom.By performing such calculations, the remaining degrees of freedom of themanipulator assembly may be more effectively controlled to cause an endeffector to appear to move along the requested degree(s) of freedom.Further, the use of such a kinematic model may advantageously reduce thecomplexity of facilitating the positioning and/or actuation of toolshaving different numbers of degrees of freedom.

In at least one embodiment, different manipulator assemblies may beconfigured to connect to the same base or support structure of therobotic surgery system. For example, different robotic arms may beconfigured to connect to the same support structure, and/or differentsurgical tools may be configured to connect to the same robotic arm. Insome cases, the same connector element on different robotic arms maycontrol different aspects of the robotic arms. For example, an uppermostconnector element on one robotic arm may control a yaw of the roboticarm, whereas the uppermost connector element on another robotic arm maycontrol a roll of the robotic arm.

To facilitate proper interpretation of signals being received by acontroller in the support structure from the robotic arms, the mappingunit may be provided to map the signals received from the robotic armsto particular inputs of a controller such as a joint space controller.For example, a common joint space controller that may be used for anumber of different manipulator assemblies may have a fixed set of inputelements. The mapping unit for a particular robotic arm may then map thesignals received from the robotic arm to the appropriate input elementsof the joint space controller. For example, the mapping unit may map asignal received from the ‘roll’ connector element of the robotic arm toa generic input element of the joint space controller. For a differentrobotic arm, the mapping unit may map a signal received from the ‘roll’connector element of the robotic arm (which may be a different connectorelement than for the first robotic arm) to the same generic inputelement of the joint space controller. In such a fashion, the same jointspace controller may be used to perform joint space calculations for anumber of different manipulator assemblies.

Similarly, manipulator assembly specific mappings may be used to mapsignals between controllers that perform calculations in different typesof spaces. For example, the support structure may, in addition to thejoint space controller, include a work space controller such as a cartspace controller that is operable to perform calculations in a, e.g.,three dimensional work space. A mapping unit may thus be provided to mapsignals output from the joint space controller to input elements of thework space controller.

In the following description, various embodiments of the presentinvention will be described. For purposes of explanation, specificconfigurations and details are set forth in order to provide a thoroughunderstanding of the embodiments. However, it will also be apparent toone skilled in the art that the present invention may be practicedwithout the specific details. Furthermore, well-known features may beomitted or simplified in order not to obscure the embodiment beingdescribed.

Referring now to the drawings, in which like reference numeralsrepresent like parts throughout the several views, FIG. 1A is anoverhead view illustration of a Minimally Invasive Robotic Surgical(MIRS) system 10, in accordance with many embodiments, for use inperforming a minimally invasive diagnostic or surgical procedure on apatient 12 who is lying down on an operating table 14. The system caninclude a surgeon's console 16 for use by a surgeon 18 during theprocedure. One or more assistants 20 may also participate in theprocedure. The MIRS system 10 can further include a patient side cart 22(surgical robot) and an electronics cart 24. The patient side cart 22may include a number of robotic arms that can each manipulate at leastone removably coupled tool assembly 26 (hereinafter simply referred toas a “tool”) through a minimally invasive incision in the body of thepatient 12 while the surgeon 18 views the surgical site through theconsole 16. An image of the surgical site can be obtained by an imagingdevice 28, such as a stereoscopic endoscope, which can be manipulated bythe patient side cart 22 so as to orient the imaging device 28. Theelectronics cart 24 can be used to process the images of the surgicalsite for subsequent display to the surgeon 18 through the surgeon'sconsole 16. The number of surgical tools 26 used at one time willgenerally depend on the diagnostic or surgical procedure and the spaceconstraints within the operating room among other factors. If it isnecessary to change one or more of the tools 26 being used during aprocedure, an assistant 20 may remove the tool 26 from the patient sidecart 22, and replace it with another tool 26 from a tray 30 in theoperating room. Further, the specific robotic arms attached to thepatient side cart 22 may also depend on the diagnostic or surgicalprocedure, and like the tools 26 can also be changed before, during, orafter a procedure.

MIRS system 10 in certain embodiments is a system for performing aminimally invasive diagnostic or surgical procedure on a patientincluding various components such as a surgeon's console 16, anelectronics cart 24, and a patient side cart 22. However, it will beappreciated by those of ordinary skill in the art that the system couldoperate equally well by having fewer or a greater number of componentsthan are illustrated in FIG. 1A. Further, computational operations orfunctions described herein as being performed by one particular elementof MIRS system 10 can be performed by other elements of MIRS system 10or, in some embodiments, distributed to two or more elements of MIRSsystem 10. For example, functions described herein as being performed byelectronics cart 24 may, in some embodiments, be performed by console 16and/or patient side cart 22. Further, it should be recognized thatmultiple elements providing the same or similar functionality may alsobe implemented within MIRS system 10. For example, MIRS system 10 mayinclude two or more consoles 16 that independently or in combinationcontrol/interact with one, two, or more patient side carts 22.Similarly, more than one electronics cart 24 may be provided (e.g., onefor each console 16), or, in some embodiments, no cart 24 may beprovided whereby the functionality described herein associated with cart24 may be distributed to one or more consoles 16, carts 22, and/or otherelements of MIRS system 10. Thus, the depiction of the system 10 in FIG.1A should be taken as being illustrative in nature, and not limiting tothe scope of the disclosure.

FIG. 1B diagrammatically illustrates a robotic surgery system 50 (suchas MIRS system 10 of FIG. 1A). As discussed above, a surgeon's console52 (such as surgeon's console 16 in FIG. 1A) can be used by a surgeon tocontrol a patient side cart (surgical robot) 54 (such as patient sidecart 22 in FIG. 1A) during a minimally invasive procedure. The patientside cart 54 can use an imaging device, such as a stereoscopicendoscope, to capture images of the procedure site and output thecaptured images to an electronics cart 56 (such as the electronics cart24 in FIG. 1A). As discussed above, the electronics cart 56 can processthe captured images in a variety of ways prior to any subsequentdisplay. For example, the electronics cart 56 can overlay the capturedimages with a virtual control interface prior to displaying the combinedimages to the surgeon via the surgeon's console 52. The patient sidecart 54 can output the captured images for processing outside theelectronics cart 56. For example, the patient side cart 54 can outputthe captured images to a processor 58, which can be used to process thecaptured images. The images can also be processed by a combination ofthe electronics cart 56 and the processor 58, which can be coupledtogether so as to process the captured images jointly, sequentially,and/or combinations thereof. One or more separate displays 60 can alsobe coupled with the processor 58 and/or the electronics cart 56 forlocal and/or remote display of images, such as images of the proceduresite, or other related images.

MIRS system 50 in certain embodiments is a system for performing aminimally invasive diagnostic or surgical procedure on a patientincluding various components such as a surgeon's console 52, anelectronics cart 56, and a patient side cart 54. However, it will beappreciated by those of ordinary skill in the art that the system couldoperate equally well by having fewer or a greater number of componentsthan are illustrated in FIG. 1B. Thus, the depiction of the system 50 inFIG. 1B should be taken as being illustrative in nature, and notlimiting to the scope of the disclosure.

FIG. 2 is a perspective view of the surgeon's console 16. The surgeon'sconsole 16 includes a left eye display 32 and a right eye display 34 forpresenting the surgeon 18 with a coordinated stereo view of the surgicalsite that enables depth perception. The console 16 further includes oneor more input control devices 36, which in turn cause the patient sidecart 22 (shown in FIG. 1A) to manipulate one or more tools. The inputcontrol devices 36 can provide the same degrees of freedom, or moredegrees of freedom, as their associated tools 26 (shown in FIG. 1A) soas to provide the surgeon with telepresence, or the perception that theinput control devices 36 are integral with the tools 26 so that thesurgeon has a strong sense of directly controlling the tools 26. To thisend, position, force, and tactile feedback sensors (not shown) may beemployed to transmit position, force, and tactile sensations from thetools 26 back to the surgeon's hands through the input control devices36.

The surgeon's console 16 is usually located in the same room as thepatient so that the surgeon may directly monitor the procedure, bephysically present if necessary, and speak to an assistant directlyrather than over the telephone or other communication medium. However,the surgeon can be located in a different room, a completely differentbuilding, or other remote location from the Patient allowing for remotesurgical procedures.

Surgeon's console 16 in certain embodiments is a device for presentingthe surgeon with information concerning the surgical site and receivinginput information from the surgeon, and includes various components suchas eyes displays and input control devices. However, it will beappreciated by those of ordinary skill in the art that the surgeon'sconsole could operate equally well by having fewer or a greater numberof components than are illustrated in FIG. 2. Thus, the depiction of thesurgeon's console 16 in FIG. 2 should be taken as being illustrative innature, and not limiting to the scope of the disclosure.

FIG. 3 is a perspective view of the electronics cart 24. The electronicscart 24 can be coupled with the imaging device 28 and can include aprocessor to process captured images for subsequent display, such as toa surgeon on the surgeon's console, or on another suitable displaylocated locally and/or remotely. For example, where a stereoscopicendoscope is used, the electronics cart 24 can process the capturedimages so as to present the surgeon with coordinated stereo images ofthe surgical site. Such coordination can include alignment between theopposing images and can include adjusting the stereo working distance ofthe stereoscopic endoscope. As another example, image processing caninclude the use of previously determined camera calibration parametersso as to compensate for imaging errors of the image capture device, suchas optical aberrations.

The electronics cart 24 in certain embodiments is a device forpresenting information concerning a surgery to a surgical team andincludes various components displays, processors, storage elements, etc.However, it will be appreciated by those of ordinary skill in the artthat the electronics cart could operate equally well by having fewer ora greater number of components than are illustrated in FIG. 3. Thus, thedepiction of the electronics cart 24 in FIG. 3 should be taken as beingillustrative in nature, and not limiting to the scope of the disclosure.

FIG. 4 shows a patient side cart 22 having a plurality of manipulatorarms 100 mounted to a support structure 110, each arm supporting asurgical instrument or tool 26 at a distal end of the manipulator arm.The patient side cart 22 shown includes four manipulator arms 100 whichcan be used to support either a surgical tool 26 or an imaging device28, such as a stereoscopic endoscope used for the capture of images ofthe site of the procedure. The support structure 110 may include one ormore elements suitable for supporting the manipulator arms 100, such aswheels, a base, legs, a spine, etc. In some embodiments, the supportstructure 110 may include electronic components such as processors,storage elements, etc., and in at least one embodiment includes aconnector for mechanically coupling the manipulator arms 100 to thesupport structure 110 and for electrically coupling components of themanipulator arms 100 and/or tools 26 (e.g., motors or other actuators)to components of the support structure 110 (e.g., the processors,storage elements, etc.).

Manipulation of the tools 26 is provided by the robotic manipulator arms100 having a number of robotic joints, where each joint provides amanipulator degree of freedom. The angle of each joint may be controlledby an actuator such as a motor or motor assembly, and in someembodiments the angle of each joint may be measured using one or moresensors (e.g., encoders, or potentiometers, or the like) disposed on orproximate to each joint. The imaging device 28 and the surgical tools 26can be positioned and manipulated through incisions in the patient sothat a kinematic remote center is maintained at the incision so as tominimize the size of the incision. Images of the surgical site caninclude images of the distal ends of the surgical instruments or tools26 when they are positioned within the field-of-view of the imagingdevice 28.

Regarding surgical tool 26, a variety of alternative robotic surgicaltools or instruments of different types and differing end effectors maybe used, with the instruments of at least some of the manipulators beingremoved and replaced during a surgical procedure. Several of these endeffectors, including DeBakey Forceps, microforceps, Potts scissors, andclip a plier include first and second end effector elements which pivotrelative to each other so as to define a pair of end effector jaws.Other end effectors, including scalpel and electrocautery probe have asingle end effector element. For instruments having end effector jaws,the jaws will often be actuated by squeezing the grip members of handle.Single end effector instruments may also be actuated by gripping of thegrip members, for example, so as to energize an electrocautery probe.

The elongate shaft of instrument 26 allows the end effectors and thedistal end of the shaft to be inserted distally into a surgical worksitethrough a minimally invasive aperture, often through an abdominal wallor the like. The surgical worksite may be insufflated, and movement ofthe end effectors within the patient will often be affected, at least inpart, by pivoting of the instrument 26 about the location at which theshaft passes through the minimally invasive aperture. In other words,manipulators 100 will move the proximal housing of the instrumentoutside the patient so that shaft extends through a minimally invasiveaperture location so as to help provide a desired movement of endeffector. Hence, manipulators 100 will often undergo significantmovement outside the patient 12 during a surgical procedure.

The patient side cart 22 in certain embodiments is a device forproviding surgical tools for assisting in a surgical procedure on apatient, and may include various components such as a support structure110, manipulator arms 100 and tools 26. However, it will be appreciatedby those of ordinary skill in the art that the patient side cart couldoperate equally well by having fewer or a greater number of componentsthan are illustrated in FIG. 4. Thus, the depiction of the patient sidecart 22 in FIG. 4 should be taken as being illustrative in nature, andnot limiting to the scope of the disclosure.

An exemplary manipulator arm in accordance with some embodiments of thepresent invention can be understood with reference to FIG. 5. Asdescribed above, a manipulator arm generally supports a distalinstrument or surgical tool and effects movements of the instrumentrelative to a base. As a number of different instruments havingdiffering end effectors may be sequentially mounted on each manipulatorduring a surgical procedure (typically with the help of a surgicalassistant), a distal instrument holder will preferably allow rapidremoval and replacement of the mounted instrument or tool. As can beunderstood with reference to FIG. 4, manipulators are proximally mountedto a base of the patient side cart. Typically, the manipulator armincludes a plurality of linkages and associated joints extending betweenthe base and the distal instrument holder. In one aspect, an exemplarymanipulator includes a plurality of joints having either redundant ornon-redundant degrees of freedom, but is lacking at least one degree offreedom necessary to fully prescribe the position (i.e., location andorientation) of the end effector.

In many embodiments, such as that shown in FIG. 5, an exemplarymanipulator arm includes a proximal revolute joint J1 that rotates abouta first joint axis so as to revolve the manipulator arm distal of thejoint about the joint axis. In some embodiments, revolute joint J1 ismounted directly to the base, while in other embodiments, joint J1 maybe mounted to one or more movable linkages or joints. The joints of themanipulator arm may be manipulated so as to control the position and/ororientation of a tool coupled thereto. In some embodiments, the jointsof the manipulator, in combination, may have redundant degrees offreedom such that the joints of the manipulator arm can be driven into arange of differing configurations for a given end effector position. Forexample, if one or more additional, redundant degrees of freedom wereadded to the manipulator arm of FIG. 5, the resulting manipulator armmay be maneuvered into differing configurations while the distalinstrument or tool 511 supported within the instrument holder 510maintains a particular state, which may include a given position orvelocity of the end effector. Regardless of whether a manipulator armincludes redundant degrees of freedom, in some embodiments the joints ofthe manipulator are not operable to independently control at least oneof the six end effector degrees of freedom that fully define theposition of the tool 511. For example, the manipulator may not beoperable to cause the tool 511 to independently roll, pitch, yaw, and/ortranslate in one or more directions.

Describing the individual links of manipulator arm 500 of FIG. 5 alongwith the axes of rotation of the joints connecting the links asillustrated in FIG. 5, a first link 504 extends distally from a pivotaljoint J2 which pivots about its joint axis and is coupled to revolutejoint J1 which rotates about its joint axis. Many of the remainder ofthe joints can be identified by their associated rotational axes, asshown in FIG. 5. For example, a distal end of first link 504 is coupledto a proximal end of a second link 506 at a pivotal joint J3 that pivotsabout its pivotal axis, and a proximal end of a third link 508 iscoupled to the distal end of the second link 506 at a pivotal joint J4that pivots about its axis, as shown. The distal end of the third link508 is coupled to instrument holder 510 at pivotal joint J5. Typically,the pivotal axes of each of joints J2, J3, J4, and J5 are substantiallyparallel and the linkages appear “stacked” when positioned next to oneanother, so as to provide a reduced width of the manipulator arm andimprove patient clearance during maneuvering of the manipulatorassembly. In many embodiments, the instrument holder 510 also includesadditional joints, such as a prismatic joint J6 that facilitates axialmovement of the instrument 511 through the minimally invasive apertureand facilitates attachment of the instrument holder 510 to a cannulathrough which the instrument 511 is slidably inserted. In someembodiments, even when combining the degrees of freedom of theinstrument holder 510 with the rest of those of manipulator arm 500, theresulting degrees of freedom are still insufficient to provide at leastone of the six degrees of freedom necessary to fully define the positionof the tool 511.

The instrument 511 may include additional degrees of freedom distal ofinstrument holder 510. Actuation of the degrees of freedom of theinstrument will often be driven by motors of the manipulator, andalternative embodiments may separate the instrument from the supportingmanipulator structure at a quickly detachable instrumentholder/instrument interface so that one or more joints shown here asbeing on the instrument are instead on the interface, or vice versa. Insome embodiments, instrument 511 includes a rotational joint J7 (notshown) near or proximal of the insertion point of the tool tip or thepivot point PP, which generally is disposed at the site of a minimallyinvasive aperture. A distal wrist of the instrument allows pivotalmotion of an end effector of surgical tool 511 about instrument jointsaxes of one or more joints at the instrument wrist. An angle between endeffector jaw elements may be controlled independently of the endeffector location and orientation. Notwithstanding these additionalkinematic degrees of freedom provided by the surgical tool 511, whichmay be considered to be part of the manipulator degrees of freedom, insome embodiments, even when combining the kinematic degrees of freedomof the surgical tool 511 with those of manipulator arm 500 (including,e.g., those of instrument holder 510), the resulting kinematic degreesof freedom are still insufficient to fully control the position of thetip of tool 511.

In a number of embodiments, the manipulator arm 500 may includeconnectors for mechanically and, in some embodiments, electrically,connecting to a support structure and a tool. In one embodiment, themanipulator arm 500 may include a support structure connector 512 thatis shaped to engage with a corresponding connector on the supportstructure. The support structure connector 512 may include one or moreelements for communicating information between elements of the supportstructure (e.g., one or more processors) and elements of the manipulatorarm 500 (e.g., motors and/or sensors). For example, the supportstructure connector 512 may include electrical and/or optical componentscoupled to the motors, sensors, and/or other elements of the manipulatorarm 500.

In another embodiment, the manipulator arm 500 may include a toolconnector 514 that is shaped to engage with a corresponding connector onthe tool. The tool connector 512 may include one or more elements forcommunicating information between elements of the tool (e.g., motors orother actuators, sensors, etc.) and elements of the manipulator arm 500(e.g., electrical and/or optical conductors in the links, electricaland/or optical components of the support structure connector 512, etc.).For example, the manipulator arm 500 may include conductors (e.g., wiresor optical fibers) arranged between and coupled to one or morecomponents of the support structure connector 512 and the tool connector512. The tool connector 512 may then also include electrical and/oroptical components for communicating information with an attached tool,thereby facilitating information to be communicated between the supportstructure and a connected tool. In some embodiments, the tool connector514 may include one or more output couplers (not shown) that maymechanically engage with corresponding input couplers of a tool, wheremovement (e.g., rotation, translation, etc.) of the output couplercauses a corresponding movement of the input coupler via the mechanicalengagement.

The manipulator arm 500 in certain embodiments is a mechanical body forholding and controlling a tool, and may include a number of links andjoints. However, it will be appreciated by those of ordinary skill inthe art that the manipulator arm could operate equally well by havingfewer or a greater number of components than are illustrated in FIG. 5.Thus, the depiction of the manipulator arm 500 in FIG. 5 should be takenas being illustrative in nature, and not limiting to the scope of thedisclosure.

FIG. 6A shows a surgical tool 600 that includes a proximal chassis 602,an instrument shaft 604, and a distal end effector 606 having a jaw 608that can be articulated to grip a patient tissue. The proximal chassis602 includes a frame 612 and, in some embodiments, a spring assembly610, that is shaped to engage with the distal end of a manipulator arm(e.g., shaped to connect to the tool connector 514 described withreference to FIG. 5). The proximal chassis 602 may also include an inputcoupler that is configured to interface with and be driven by an outputcoupler of the manipulator arm. The input coupler is drivingly coupledwith an input link of a spring assembly 610. The spring assembly 610 ismounted to the frame 612 of the proximal chassis 602 and includes anoutput link that is drivingly coupled with a drive shaft that isdisposed within the instrument shaft 604. The drive shaft is drivinglycoupled with the jaw 608. In some embodiments, the proximal chassis 602may also include electrical and/or optical elements for electricallyand/or optically coupling to corresponding elements of the manipulatorarm (e.g., corresponding elements of the tool connector 514). In thisfashion, information may be communicated between elements of the tool600 (e.g., motors, actuators, sensors, etc.) and elements of themanipulator arm.

In accordance with some embodiments and as shown in FIG. 6A, thesurgical tool 600 may not include any degrees of freedom for altering aposition of the end effector 606. In other embodiments, the surgicaltool 600 may include one or more joints for adding degrees of freedomfor altering the position of the end effector 606. For example, theinstrument shaft 604 may include joints for changing a pitch and/or yawof the end effector 606. Further, in some embodiments and as shown inFIG. 6A, the surgical tool 600 may include one or more degrees offreedom for actuating the end effector 606. For example, the springassembly 610 may be operable to actuate the jaw 608. Additionalcharacteristics of surgical tool 600, as well as other surgical tools,are described in commonly-owned U.S. patent application Ser. No.13/297,158, filed Nov. 15, 2011 (now U.S. Pat. No. 9,095,362), entitled“Method for Passively Decoupling Torque Applied By a Remote ActuatorInto an Independently Rotating Member,” the disclosure of which isincorporated herein by reference in its entirety.

FIG. 6B illustrates a wristed endoscope 620 that may, in someembodiments, be used in robotic minimally invasive surgery. Theendoscope 620 includes an elongate shaft 622 and a flexible wrist 624located at the working end of the shaft 622. A housing 626 allows thesurgical instrument 620 to releasably couple to a manipulator located atthe opposite end of the shaft 624. An endoscopic camera lens isimplemented at the distal end of the flexible wrist 624. A lumen (notshown) runs along the length of the shaft 622 which connects the distalend of the flexible wrist 624 to the housing 626. In a “fiber scope”embodiment, imaging sensor(s) of the endoscope 620, such as chargecoupled devices (CCDs), may be mounted inside the housing 626 withconnected optical fibers running inside the lumen along the length ofthe shaft 622 and ending at substantially the distal end of the flexiblewrist 624. In an alternate “chip-on-a-stick” embodiment, the imagingsensor(s) of the endoscope 620 may be mounted at the distal end of theflexible wrist 624. The imaging sensor(s) may be two-dimensional orthree-dimensional.

In some embodiments, the flexible wrist 624 may have at least one degreeof freedom to allow the endoscope 620 to articulate and maneuver easilyaround internal body tissues, organs, etc. to reach a desireddestination (e.g., epicardial or myocardial tissue). The housing 626 mayhouse a drive mechanism for articulating the distal portion of theflexible wrist 624. The drive mechanism may be cable-drive, gear-drive,belt drive, or another type of drive mechanism suitable to drive theflexible wrist 624 along its degree(s) of freedom. For example, in oneembodiment, the flexible wrist 624 may have two translation degrees offreedom and the shaft 622 may be operable to rotate around an axis alongthe length of the shaft 622. In some medical procedures, the articulateendoscope 620 maneuvers and articulates around internal organs, tissues,etc. to acquire visual images of hard-to-see and/or hard-to-reachplaces. Additional characteristics of the endoscope 620, as well asother surgical tools, are described in commonly-owned U.S. patentapplication Ser. No. 11/319,011, filed Dec. 27, 2005 (published as U.S.Patent Application Publication No. 2006-0178556), entitled “Articulateand Swapable Endoscope for a Surgical Robot,” the disclosure of which isincorporated herein by reference in its entirety.

In at least one embodiment, the housing 626 may be shaped to engage withthe distal end of a manipulator arm (e.g., shaped to connect to the toolconnector 514 with reference to FIG. 5). Further, the housing 626 mayinclude electrical and/or optical elements for electrically and/oroptically coupling to corresponding elements of the manipulator arm. Inthis fashion, information may be communicated between elements of thetool 620 (e.g., motors, actuators, sensors, etc.) and elements of themanipulator arm. FIG. 6C is a perspective view of the distal end of anovertube with suction ports. The overtube 630 defines an instrumentlumen 632 which extends through the overtube 630 to permit passage of aninstrument. The overtube 630 further comprises one or more suctionpassages 634 which are coupled to a vacuum source. The overtube 630 may,in various embodiments, be formed out of any of a variety of materialssuitable for surgical use and may be provided with any of variety ofstiffnesses. For example, the overtube 630 may comprise a substantiallyrigid material, may comprise a flexible material, or may comprise acombination of one or more substantially rigid portions and one or moreflexible portions to provide a bendable structure. The cross-sectionalshape of the overtube 630 may also vary. In the illustrated embodiment,the overtube 630 has a substantially circular cross-sectional shape andis made out of polyurethane. In other embodiments, other cross-sectionalshapes may be used, such as, e.g., oval, rectangular, triangular, etc.,depending on the application.

In the illustrated embodiment, the suction passages 634 comprises aplurality of vacuum lumens within the wall of the overtube 630, witheach vacuum lumen being coupled to the vacuum source (not shown). Thevacuum source may be operated to create a vacuum pressure in eachsuction passage 634, thereby creating a suction force onto a tissuesurface which the suction passages 634 are in contact with. As a resultof this suction force, the overtube 630 will be attached to the tissuesurface. If the vacuum pressure is discontinued, the tissue surface willbe released and the overtube 630 will no longer be attached to thetissue. Accordingly, by controllably providing a suction force via thesuction passages 634, the overtube 630 can be releasably attached topatient's tissue surface. A surgical instrument, such as an irrigationtool, cutting tool, etc., may then be inserted through the instrumentlumen 200 to treat tissue disposed within the instrument lumen 632.

In accordance with some embodiments, the overtube 630 may be made ofsubstantially rigid material and not include any degrees of freedom foraltering a position of the overtube 630. In other embodiments, theovertube 630 may include one or more joints for adding degrees offreedom for altering the position of the distal end of the overtube 630.For example, the overtube 630 may include joints for changing a pitchand/or yaw of the distal end of the overtube 630. Further, in someembodiments, the overtube 630 may include one or more degrees of freedomfor actuating functionality of the overtube 630. For example, a vacuumsource (not shown) may be operable to create or remove a vacuum pressurein one or more suction passages 634. Additional characteristics of theovertube 630, as well as other surgical tools, are described incommonly-owned U.S. patent application Ser. No. 11/618,374, filed Dec.29, 2006 (published as U.S. Patent Application Publication No.2008-0108871), entitled “Vacuum Stabilized Overtube for EndoscopicSurgery,” the disclosure of which is incorporated herein by reference inits entirety.

Further, in at least one embodiment, the overtube 630 may be provided inor coupled to a housing (not shown) that may be shaped to engage withthe distal end of a manipulator arm (e.g., shaped to connect to the toolconnector 514 with reference to FIG. 5). Further, the housing mayinclude electrical and/or optical elements for electrically and/oroptically coupling to corresponding elements of the manipulator arm. Inthis fashion, information may be communicated between elements of theovertube 630 (e.g., motors, actuators, sensors, etc.) and elements ofthe manipulator arm.

FIG. 6D illustrates a non-wristed endoscope 640 that may, in someembodiments, be used in robotic minimally invasive surgery. Thenon-wristed endoscope 640 is similar to the wristed endoscope 620depicted in and discussed with reference to FIG. 6B, and thus similarlyincludes a housing 646 and a shaft 622. The difference is that thenon-wristed endoscope 640 does not include a flexible wrist. Thenon-wristed endoscope has a reduced number of degrees of freedomcompared to the wristed endoscope, and in this particular example,non-wristed endoscope 640 does not have a wrist pitch or wrist yaw.

The surgical tool 600, endoscope 620, and overtube 30 are various toolsthat include a variety of components. However, it will be appreciated bythose of ordinary skill in the art that these tools could operateequally well by having fewer or a greater number of components than areillustrated in FIGS. 6A to 6C. Further, it would will also beappreciated that other tools may also or alternatively be used, such asgripping devices, electrosurgical paddles, vacuums, irrigators,staplers, scissors, knifes, etc. Thus, the depiction of surgical toolsin FIGS. 6A to 6C should be taken as being illustrative in nature, andnot limiting to the scope of the disclosure.

FIG. 7A is a perspective view of a master control input device 700 thatmay be part of a surgeon's console 16 (FIG. 1A) in accordance with anembodiment. The master control 700 includes a gimbal or wrist 720 thatis operatively coupled to an articulated arm 740.

Master control input device 700 has a number of degrees of freedom andis operable to control a manipulator assembly (e.g., manipulator arm 500of FIG. 5). The degrees of freedom of input device 700 includeskinematic degrees of freedom defined by joints of input device 700, usedto control the kinematics of manipulator arm 500, and may also includeactuation degrees of freedom used to actuate a tool (e.g., instrument511) connected to manipulator arm 500. Input device 700, like a tool ofmanipulator arm 500, may also be considered to have an end effector (or,more generally, a control frame) associated therewith, which itself hasa number of kinematic degrees of freedom.

In some embodiments, input device 700 may have a sufficient number ofdegrees of freedom to fully control the position of an end effector. Forexample, the input device 700 may have six degrees of freedom that mayindependently control the three translation and three orientationdegrees of freedom of an end effector of the instrument 511. In somecases, even though the input device 700 has such a sufficient number ofdegrees of freedom, the manipulator assembly (e.g., manipulator arm 500)has a number of degrees of freedom that is insufficient to independentlycontrol the three translation and three orientation degrees of freedomof the end effector. For example, the manipulator arm 500 may have onlyfive degrees of freedom.

In some embodiments, the input device 700 may have additional degrees offreedom, which may be degrees of freedom operable to control theposition of the end effector (e.g., a redundant degree of freedom),and/or may be degrees of freedom operable to actuate the instrument 26(e.g., turning on or off suction or irrigation, actuating a clamp,engaging a staple with tissue, etc.). An input device having additionaldegrees of freedom is described in commonly-owned U.S. patentapplication Ser. No. 10/121,283, filed Apr. 11, 2002 (now U.S. Pat. No.6,684,129), entitled “Master Having Redundant Degrees of Freedom,” thedisclosure of which is incorporated herein by reference in its entirety.Further, in at least one embodiment, the instrument 511, either alone orin combination with a manipulator arm 500, may have additional kinematicdegrees of freedom that add to the degrees of freedom of the manipulatorarm 500. For example, the instrument 511 may have joints for controllingthe position of the end effector. In some cases, even when combining thekinematic degrees of freedom of the manipulator arm 500 with thekinematic degrees of freedom of the instrument, the position of the endeffector may not be fully controlled. This may be, e.g., due to thejoints of the instrument 511 merely adding kinematic degrees of freedomthat are redundant to those already provided by the manipulator arm 500.In some embodiments, the instrument 511 may have additional actuationdegrees of freedom for actuating the instrument 511 (e.g., turning on oroff suction or irrigation, actuating a clamp, engaging a staple withtissue, etc.).

To facilitate control of the instrument 511, the master control inputdevice 700 may include one or more actuators or motors and, in someembodiments, sensors for each of a plurality of joints of the mastercontrol input device 700. The motors and sensors of the input device 700may be operatively linked to the motors and sensors associated with themanipulator arms (e.g., arm 500 of FIG. 5) and the surgical instrumentsmounted thereon (e.g., instrument 511 of FIG. 5) via a control systemdisposed in, e.g., the surgeon's console 16, the electronics cart 24,and/or the patient cart 22, and/or any other element of MIRS system 10(FIG. 1). The control system may include one or more processors foreffecting control between the master control device input and responsiverobotic arm and surgical instrument output and for effecting controlbetween robotic arm and surgical instrument input and responsive mastercontrol output in the case of, e.g., force feedback.

FIG. 7B is a perspective view of a gimbal or wrist 720 according to anembodiment. According to this embodiment, gimbal or wrist 720 allowsrotation of an actuatable handle 722 about three axes, axis 1, axis 2,and axis 3. More specifically, the handle 722 is coupled to a firstelbow-shaped link 724 by a first pivotal joint 726. The first link 724is coupled to a second elbow-shaped link 728 by a second pivotal joint730. The second link 728 is pivotally coupled to a third elbow-shapedlink 732 by a third pivotal joint 734. The gimbal or wrist 720 may bemounted on an articulated arm 740 (as shown in FIG. 7A) at axis 4 suchthat the gimbal or wrist 720 can displace angularly about axis 4. By wayof such links and joints, the gimbal or wrist 720 may provide a numberof kinematic degrees of freedom for the control input device 700 and beoperable to control one or more of the end effector degrees of freedom.

In some embodiments, the handle 722 may include a pair of grip members723 for actuating a tool or end effector. For example, by opening orclosing the grip members 723, the jaw 608 of the end effector 606 (FIG.6) may similarly be opened or closed. In other embodiments, one or moreinput elements of the handle 722 and/or of other elements of thesurgeon's console 16 may be operable to actuate one or more degrees offreedom of the instrument 511 other than degrees of freedom forcontrolling the position of the instrument 26. For example, thesurgeon's console 16 may include a foot pedal coupled to the controlsystem for activating and deactivating a vacuum pressure.

In some embodiments, the joints of the gimbal or wrist 720 may beoperatively connected to actuators, e.g., electric motors, or the like,to provide for, e.g., force feedback, gravity compensation, and/or thelike. Furthermore, sensors such as encoders, potentiometers, and thelike, may be positioned on or proximate to each joint of the gimbal orwrist 720, so as to enable joint positions of the gimbal or wrist 720 tobe determined by the control system.

FIG. 7C is a perspective view of an articulated arm 740 according to anembodiment. According to this embodiment, the articulated arm 740 allowsrotation of a gimbal or wrist 720 (FIG. 7B) about three axes, axis A,axis B, and axis C. More specifically, the gimbal or wrist 720 may bemounted on the arm 740 at axis 4 as previously described with referenceto FIG. 7B. The gimbal or wrist 720 is coupled to a first link 742 whichis pivotally coupled to a second link 744 by a first pivotal joint 746.The second link 744 is pivotally coupled to a third link 748 by a secondpivotal joint 750. The third link 748 may be pivotally coupled to thesurgeon's console 16 (FIG. 1) by a third pivotal joint 752. By way ofsuch links and joints, the articulated arm 740 may provide a number ofkinematic degrees of freedom for the control input device 700 and beoperable to control one or more of the kinematic degrees of freedom of amanipulator assembly to thereby control the position of an instrument(e.g., instrument 511 of FIG. 5).

In some embodiments, the joints of the articulated arm 740 may beoperatively connected to actuators, e.g., electric motors, or the like,to provide for, e.g., force feedback, gravity compensation, and/or thelike. Furthermore, sensors such as encoders, potentiometers, and thelike, may be positioned on or proximate to each joint of the articulatedarm 740, so as to enable joint positions of the articulated arm 740 tobe determined by the control system.

Input device 700 in certain embodiments is a device for receiving inputsfrom a surgeon or other operator and includes various components such asa gimbal or wrist 720 and an articulated arm 740. However, it will beappreciated by those of ordinary skill in the art that the input devicecould operate equally well by having fewer or a greater number ofcomponents than are illustrated in FIGS. 7A to 7C. Thus, the depictionof the input device 700 in FIGS. 7A to 7C should be taken as beingillustrative in nature, and not limiting to the scope of the disclosure.

FIG. 8A is part of a robotic system including a manipulator assembly 810and a support structure 850 according to a first embodiment. Themanipulator assembly 810 includes a manipulator 820 and a tool 840,where the manipulator 820 is disposed between and may be connected toeach of the support structure 850 and the tool 840.

The manipulator 820 includes a plurality of links 822 coupled togetherby joints 824. The manipulator 820 may also include a number ofactuators 826 such as motors, and a number of sensors 828 such aspotentiometers, where each joint may be associated with an actuatorand/or a sensor. The actuator may be operable to control a degree offreedom of the manipulator, such as by controlling one or more joints torotate, translate, etc. Further, the sensor may be operable to measure aposition or state of each of a corresponding joint.

The manipulator 820 also includes a connector 830 at a proximal end thatis shaped to engage a corresponding connector of the support structure850, the connector 830 forming a mechanical interface operable tomechanically couple the manipulator arm 820 to the support structure.The connector 830 may also include electrical and/or optical contacts832 that are positioned, sized, and shaped to engage correspondingcontacts of the support structure 850, whereby the electrical and/oroptical contacts 832 may form an electrical interface operable tocommunicate information between elements of the manipulator assembly 810and elements of the support structure 850. The contacts 832 may bedirectly or indirectly coupled (e.g., electrically, optically, etc.) toelements of the manipulator 820 such as actuators 826 and/or sensors828. In one embodiment, each contact 832 is coupled to one actuator 826or sensor 828. Accordingly, information may be communicated betweenelements of the manipulator 820 (e.g., the actuators and sensors 828)and elements of the support structure 850. For example, instructions maybe communicated from the support structure to the motors 826 to causethe motors to control corresponding degrees of freedom of themanipulator arm 820, and position/state information may be communicatedfrom the sensors 828 to the elements of the support structure 850.

In some embodiments, the manipulator 820 may also include a connector834 at a distal end that is shaped to engage a corresponding connectorof the tool 840, the connector 834 forming a mechanical interfaceoperable to mechanically couple the manipulator arm 820 to the tool 840.The connector 834 may also include one or more mechanical elements 836that may be sized and shaped to engage with corresponding mechanicalelements 842 included in the tool 840. The mechanical element 836 may beactuated or otherwise controlled by an actuator 826, such that uponengaging the tool 840 with the manipulator 820, actuation of themechanical element 836 may cause actuation of the correspondingmechanical element 842 in the tool 840. The corresponding mechanicalelement 842 in the tool 840 may be operable to manipulate a degree offreedom of the tool 840 (e.g., actuating an end effector 844 of the tool840. In some embodiments, one or more sensors 828 included in ordisposed on the manipulator 820 may be operable to sense a positionand/or state of the tool 840.

Support structure 850 includes a connector 852 that is shaped to engagea corresponding connector (e.g., connector 830) of the manipulator 820.The connector 852 may also include electrical and/or optical contacts854 that are positioned, sized, and shaped to engage correspondingcontacts (e.g., contacts 832) of the manipulator 820, whereby theelectrical and/or optical contacts 854 may form an electrical interfaceoperable to communicate information between elements of the manipulatorarm 820 and elements of the tool 840. The contacts 854 may be directlyor indirectly coupled (e.g., electrically, optically, etc.) to elementsof the support structure 850 such as a hardware mapping unit 856, ajoint space controller 858, a joint space work space mapping unit 860,and/or a work space controller 862.

In one embodiment, the hardware mapping unit 856 is operable to mapsignals between a manipulator assembly 810 (e.g., signals to/fromactuators, sensors, etc. of the manipulator assembly via, e.g.,electrical/optical contacts 854) and joint space controller 858. Thehardware mapping unit 856 may include (and/or acquire) a specific mapfor each of a plurality of different manipulator assemblies, such asdifferent manipulator arms 820 and/or different tools 840. In someembodiments, the hardware mapping unit 856 may include input maps andoutput maps. In one embodiment, the input maps and output maps may bedifferent from one another. For example, an actuator 826 and a sensor828 may each communicate with the support structure 850 via a singlecontact 854. Accordingly, the input map may map signals from thecontacts 854 corresponding to sensors 828 to the joint space controller858, and the output map may map signals from the joint space controller858 to the contacts 854 corresponding to actuators 826. In anotherembodiment, the input maps and output maps may be the same as oneanother. For example, an actuator 826 and a sensor 826 may bothcommunicate with the support structure 850 via a single contact 854.Accordingly, the input map and output map may be the same as they maymap input and output interface elements of the joint space controller858 to the same contact 854.

The joint space controller 858 may be a processor operable to performcalculations and execute a number of algorithms in joint space. Forexample, the joint space controller 858 may be operable to execute jointmotion algorithms, comparisons of redundant sensors on each joint, motorhealth algorithms, etc.

The joint space controller 858 may receive input information from anumber of different sources. For example, the joint space controller 858may receive outputs from the work space controller 862 via the jointspace work space mapping unit 860. For another example, the joint spacecontroller 858 may receive inputs (e.g., sensor signals from sensors826) from the manipulator 820 via the hardware mapping unit 856. For yetanother example, the joint space controller 858 may receive inputs froman input device such as a master control input device (FIGS. 7A to 7C).Further, the joint space controller 858 may provide output informationto a number of different destinations. For example, the joint spacecontroller 858 may output information (e.g., control information) to themanipulator 820 (e.g., to actuators 826) via the hardware mapping unit856. For another example, the joint space controller 858 may provideoutputs to the work space controller 865 via the joint space work spacemapping unit 860 for further processing. For yet another example, thejoint space controller 858 may provide outputs to an input device suchas a master control input device (FIGS. 7A to 7C).

The joint space work space mapping unit 860 is operable to map signalsbetween the joint space controller 858 (e.g., signals representingdesired or actual positions or states of the degrees of the freedom ofthe manipulator assembly 810) and the work space controller 862. Thejoint space work space mapping unit 860 may include a specific map foreach of a plurality of different manipulator assemblies, such asdifferent manipulator arms 820 and/or different tools 840. Accordingly,the joint space work space mapping unit 860 may be operable to mapinput/output interface elements of the joint space controller 858 toinput/output interface elements of the work space controller 862 basedon the particular manipulator assembly 810 connected to the supportstructure 850.

The work space controller 862 may be a processor operable to performcalculations and execute a number of algorithms in a multi-dimension(e.g., 3 dimensional) work space. This may be, e.g., a work space basedon polar coordinates, Cartesian coordinates, or other type of coordinatesystem. The work space controller 862 may be operable to execute avariety of algorithms. For example, the work space controller 865 may beoperable to perform forward kinematics, inverse kinematics, determinethe work space error between desired and actual positions/orientations,etc.

The work space controller 862 may receive input information from anumber of different sources. For example, the work space controller 862may receive outputs from the joint space controller 858 via the jointspace work space mapping unit 860. For another example, the work spacecontroller 862 may receive inputs from an input device such as a mastercontrol input device (FIGS. 7A to 7C). Further, the work spacecontroller 862 may provide output information to a number of differentdestinations. For example, the work space controller 862 may outputinformation (e.g., control information) to joint space controller 858via the joint space work space mapping unit 860. For yet anotherexample, the work space controller 862 may provide outputs to an inputdevice such as a master control input device (FIGS. 7A to 7C).

In at least one embodiment, a plurality of different tools may beprovided, having the same or different number of degrees of freedom. Thetools may all have the same mechanical interface so that they may all beattached to the same manipulator arm and, in some cases, switched duringa surgical procedure. Further, a plurality of different manipulator armsmay be provided, having the same or different degrees of freedom. Themanipulator arms may also have the same mechanical interfaces so thatthey may all be attached to the same support structure and to the sametools and, in some cases, may be switched during or between surgicalprocedures.

In one embodiment, the controllers and/or mapping units described hereinmay be individual processors coupled to individual or a commoninformation storage element. In other embodiments, the controllersand/or mapping units described herein may be implemented as softwarecode on a tangible non-transitory computer readable storage medium.Although these elements are described as being part of support structure850, in some embodiments some or all of these elements may be includedin other parts of the robotic system, such as in a control systemdisposed in, e.g., the manipulator 820, the surgeon's console 16, theelectronics cart 24, and/or the patient cart 22, and/or any otherelement of MIRS system 10 (FIG. 1).

FIG. 8B is part of a robotic system including a manipulator assembly 810and a support structure 850 according to a second embodiment. Themanipulator assembly 810 and support structure 850 in this embodimentare similar to that discussed with reference to FIG. 8A, and thus thestructures and functions discussed with reference to FIG. 8A are equallyapplicable for this embodiment.

In accordance with this embodiment, however, the tool 840 includes anelectrical/optical contact 846 coupled to and operable to communicateinformation with an internal element 848 of the tool 840. The element848 may be, e.g., an actuator such as actuator 826 and/or a sensor suchas sensor 828. The actuator may be operable to control a degree offreedom of the tool 840, and the sensor 828 may be operable to sense aposition and/or state of the degree of freedom. For example, theactuator may be operable to control a joint of the tool 840, and thesensor may be operable to sense a position of the joint. In someembodiments, the tool may include a number of actuators and/or sensorsfor controlling a number of degrees of freedom of the tool 840. In atleast one embodiment, the same electrical/optical contact 846 in eachtool 840, that is, the contact that couples to contact 837 may beoperable to control different degrees of freedom in different tools. Forexample, in one embodiment, contacts 837 and 846 may be operable tocontrol a roll of a first tool, whereas contacts 837 and 846 may beoperable to control a pitch of a second tool.

Further, the connector 834 of the manipulator arm 820 may also includean electrical/optical contact 837 for electrically/optically engagingthe contact 846 of the tool 840 so as to facilitate communicationbetween the tool 840 and the manipulator arm 820. Similar to actuators826 and sensors 828 of the manipulator arm 820, the contact 837 may becoupled to one or more contacts 832 of the connector 830 at the proximalend of the manipulator arm 820 such that information can be communicatedbetween the tool 840 and the support structure 850.

FIG. 8C is part of a robotic system including a manipulator assembly 810and a support structure 850 according to a third embodiment. Themanipulator assembly 810 and support structure 850 in this embodimentare similar to that discussed with reference to FIG. 8A, and thus thestructures and functions discussed with reference to FIG. 8A are equallyapplicable for this embodiment.

The tool 840 in this embodiment includes a tool identification unit 845.The tool identification unit 845 may store a tool identifier thatidentifies one or more characteristics of the tool. For example, thetool identifier may identify the type of tool (e.g., endoscope, jaws,vacuum, electrosurgical paddle, etc.), the number of degrees of freedomof the tool (e.g., 3 degrees of freedom, 4 degrees of freedom, 5 degreesof freedom, etc.), the types of degrees of freedom (e.g., motion degreesof freedom such as roll, translate, etc., actuation degrees of freedomsuch as activating a vacuum or electrode, etc.), etc. The toolidentification unit 845 may be operable to communicate the toolidentifier to other elements of the robotic system, such as elements ofthe support structure. The tool identifier may then be used for avariety of purposes, such as for configuring the hardware mapping unit845 and/or the joint space work space mapping unit 860.

The tool identification unit 845 may be implemented using one or more ofa variety of techniques. For example, the tool identification unit 845may be coupled to a contact (e.g., contact 846) of the tool 840 and maythus be operable to communicate the tool identifier via one or morewires. For another example, the tool identification unit 845 may beoperable to wirelessly communicate the tool identifier. For example, thetool identification unit 845 may be a radio frequency identification(RFID) chip having an identifier associated with one or morecharacteristics of the tool 840. Accordingly, the manipulator arm 820may include an RFID reader 839 for reading the tool identifier when thetool 840 is within range of the manipulator arm 820. The RFID reader 839may be coupled to one or more contacts 832 of the connector 830 suchthat the RFID reader 839 may be operable to communicate the toolidentifier to other elements of the robotic system such as elements ofthe support structure 850.

In other embodiments, other wireless techniques may be used tocommunicate a tool identifier from the tool 840 to other elements of therobotic system. For example, the tool 840 and other elements of therobotic system (e.g., manipulator 820, support structure 850, or otherelements) may include Bluetooth™ circuitry, wireless communicationcircuitry for communicating in accordance with IEEE 802.11 standards,the IrDA standard, or other wireless communication standards.

Further, in some embodiments, the manipulator arm 820 may also oralternatively include manipulator identification unit (not shown) thatmay store a manipulator identifier that identifies one or morecharacteristics of the manipulator and/or a connected tool. For example,the manipulator identifier may identify the type of manipulator (e.g.,parallelogram), the number of degrees of freedom of the manipulatorand/or a connected tool (e.g., 3 degrees of freedom, 4 degrees offreedom, 5 degrees of freedom, etc.), the types of degrees of freedom ofthe manipulator and/or a connected tool (e.g., motion degrees of freedomsuch as roll, translate, etc., actuation degrees of freedom such asactivating a vacuum or electrode, etc.), etc. The manipulatoridentification unit may be operable to communicate the manipulatoridentifier to other elements of the robotic system, such as elements ofthe support structure. The manipulator identifier may then be used for avariety of purposes, such as for configuring the hardware mapping unit845 and/or the joint space work space mapping unit 860. Like the toolidentification unit 845, the manipulator identification unit may beoperable to communicate the manipulator identifier to other elements ofthe robotic system via wired means (e.g., through a contact 832) orwireless means (e.g., via an RFID tag coupled to the manipulator arm 820and an RFID reader coupled to the support structure 850).

Manipulator assembly 810 and support structure 850 in certainembodiments include various components such actuators, sensors, joints,tools, connectors, mapping units, and controllers. However, it will beappreciated by those of ordinary skill in the art that the input devicecould operate equally well by having fewer or a greater number ofcomponents than are illustrated in FIGS. 8A to 8C. It will be furtherappreciated by those of ordinary skill in the art that the elementsdescribed with reference to each of FIGS. 8A to 8C could be usedsimultaneously or separately from one another. For example, theidentification systems described with reference to FIG. 8C could be usedwith the system described with reference to FIG. 8B. Thus, the depictionof the manipulator assembly 810 and support structure 850 in FIGS. 8A to8C should be taken as being illustrative in nature, and not limiting tothe scope of the disclosure.

FIG. 9A illustrates a mapping between connector input/output elementsand joint space interface elements according to a first embodiment. Themapping as shown is illustrated as a first mapping between connectorinput elements 910, connector output elements 920, and joint spaceinterface elements 930.

The connector input elements 910 are, in some embodiments, part of alayer of hardware devices, such as elements for receiving sensor signalsof a manipulator assembly. For example, connector input elements 910 maycorrespond to electrical and/or optical contacts (e.g., contacts 854)designated to receive information from devices such as sensory deviceslocated in a manipulator assembly (e.g., sensors 828). Accordingly, eachcontact may be operable to receive information from elements of themanipulator assembly.

In one embodiment, one or more of the connector input elements 910receives a sensor signal 912. The sensor signal 912 may indicate thejoint state of a connected manipulator assembly. For example, the sensorsignal 912 may be information received from a sensor 828 that measures aposition or state of a degree of freedom of the manipulator assembly810. The degree of freedom may correspond to a degree of freedom of themanipulator arm 820 or, in some embodiments, to a degree of freedom of aconnected tool 840.

In the example illustrated in FIG. 9A, the sensor signal 912 indicatesthe state or position of an outer yaw (A1) of the manipulator arm. Othersensor signals, such as sensor signal 914, may indicate the state orposition of different joints of the connected manipulator assembly. Forexample, the sensor signal 914 in this example indicates the state orposition of an outer pitch (A2) of the manipulator arm. Yet other sensorsignals may indicate the state or position of even more joints of theconnected manipulator assembly, such as an outer roll (A3), an insertionjoint (A4), and an instrument roll (A5). It should be recognized thatparticular joint being indicated by the sensor signal received at theconnector input elements 910 is, in many embodiments, determined by themanipulator assembly 810. That is, the sensor signal received at theconnector input elements 910 may correspond to sensor signals providedat corresponding contacts (e.g., electrical and/or optical contacts 832)on the attached manipulator assembly, where the sensor signals providedat the contacts of the manipulator assembly may indicate differentjoints or degrees of freedom for different manipulator arms and/ortools. In some embodiments, the different joints or degrees of freedomof a manipulator assembly may result from changing a tool of themanipulator assembly.

The sensor signals provided at the connector input elements 910 are thencommunicated to the joint space interface elements 930 via a firstmapping 940. The first mapping 940 may be generated by, for example, thehardware mapping unit 856, and may operate to map the received sensorsignals to particular joint space interface elements 930.

The joint space interface elements 930 may include one or more elementshaving fixed or otherwise predefined characteristics. For example, thejoint space interface elements 930 may be defined to each correspond toa particular degree of freedom of one or more manipulator assemblies(i.e., manipulator arms and/or tools). In the example illustrated inFIG. 9A, the joint space interface elements 930 are predefined tocorrespond to, e.g., a secondary outer yaw (J1), an outer yaw (J2), anouter pitch (J3), outer roll (J4), an insertion joint (J5), aninstrument roll (J6), and additional joints (J7) to (J10).

In some embodiments, the joint space interface elements 930 may operateas input/output elements of an algorithm being executed by a processoror, in some embodiments into a controller such as joint space controller858. Accordingly, the joint space controller 858 may be operable toprocess the sensor signals in joint space by, for example, executing oneor more algorithms that loop over individual joints.

The joint space controller 858 may then be operable to output theresults via the same or different joint space interface elements 930. Inthe example illustrated in FIG. 9A, the joint space controller 858outputs the results using the same joint space interface elements 930.The output signals may be, for example, control signals for controllingone or more elements of the manipulator assembly such as actuators 826.The joint space interface elements 930 may be mapped to one or moreconnector output elements 920 via a second mapping 950. The secondmapping 950 may be generated by, for example, the hardware mapping unit856, and may operate to map the outputs of the joint space interfaceelements 930 to the connector output elements 920.

The connector output elements 920 are, in some embodiments, part of alayer of hardware devices, such as elements for sending control signalsto a manipulator assembly. For example, connector output elements 920may correspond to electrical and/or optical contacts (e.g., contacts854) designated to send information to devices such as actuators locatedin a manipulator assembly (e.g., actuators 826). Accordingly, eachcontact may be operable to communicate information to elements of themanipulator assembly. In the example shown in FIG. 9A, a control signal922 is output to an actuator of the manipulator assembly 810 forcontrolling an outer yaw of the manipulator assembly 810, whereas acontrol signal 924 is output to an actuator of the manipulator assembly810 for controlling an outer pitch of the manipulator assembly 810.

In some embodiments, the connector input elements 910 and connectoroutput elements 920 are separate from one another, whereas in otherembodiments the connector input elements 910 and connector outputelements 920 may be the same as one another. In the example shown inFIG. 9A, the connector output elements 920 are different than theconnector input elements 910, indicating that some contacts 832 may beused for receiving sensor signals from sensors 828 whereas othercontacts 832 may be used for sending control signals to actuators 826.However, in other embodiments, the connector output elements 920 may bethe same as connector input elements 910, indicating that some contacts832 may be used for both receiving sensor signals from sensors 828 andsending control signals to actuators 826.

Returning to the first mapping 940 and the second mapping 950, it shouldbe apparent that these mappings operate to map signalsreceived/communicated to the connector input/output elements to and fromthe appropriate joint space interface elements. In many embodiments,these mappings are different for different manipulator assemblies. Forexample, in the embodiment illustrated in FIG. 9A, the first connectorinput element 910 a does not receive any signals from the manipulatorassembly, whereas the second connector input element 910 b may receive asensor signal indicating a position or state of an outer yaw of themanipulator assembly. The first mapping 940 is then customized to thatparticular manipulator assembly, and operates to map the secondconnector input element 910 b to the second joint space (e.g., jointspace interface elements 930) interface element 930 b, which the jointspace controller 858 assumes and processes as an outer yaw degree offreedom. The first mapping 940 does not map any signal to the firstjoint space interface element 930 a, since the manipulator assembly doesnot have any degree of freedom corresponding to a secondary outer yawjoint.

Turning briefly to FIG. 9B, FIG. 9B illustrates a mapping betweenconnector input/output elements and joint space interface elementsaccording to a second embodiment. This embodiment is similar to thatdiscussed with reference to FIG. 9A, except in this case the supportstructure 850 is coupled to a different manipulator assembly 820 (themanipulator assembly may be different as a result of changing a tool ofthe assembly) such that the sensor signal 912 is not received at thesecond connector input element 910 b but rather is received at the thirdconnector input element 910 c. Since the joint space interface elements930 do not change with different manipulator assemblies, the firstmapping 940 must change so that the sensor signals received at theconnector input elements 910 are mapped to the appropriate joint spaceinterface elements 930. In the example of FIG. 9B, the first mapping 941that corresponds to the new manipulator assembly maps the thirdconnector input element 910 c to second joint space interface element930 b, rather than to the third joint space interface element 930 c asit did in the example of FIG. 9A. Similarly, the example in FIG. 9B mayuse a different second mapping 951 than that used in the example in FIG.9A. By using different mappings for different manipulator assemblies,advantageously the same joint space controller 858 may be used fordifferent manipulator assemblies.

FIG. 9C illustrates a mapping between joint space interface elements andwork space interface elements according to an embodiment. The jointspace interface elements 960 in this embodiment are mapped to work spaceinterface elements 970 via a joint space work space mapping 980. Thejoint space interface elements 960 may be similar to those discussedwith reference to FIGS. 9A and 9B, but in this embodiment may beoperable to communicate information between the joint space controller858 and the work space controller 862.

In some embodiments, the work space interface elements 970 may operateas input/output elements of an algorithm being executed by a processoror, in some embodiments, into a controller such as work space controller862. Accordingly, the work space controller 862 may be operable toprocess signals output by the joint space controller 858 in a work spaceby receiving those signals via the work space interface elements 970. Inat least one embodiment, the work space controller 862 corresponds to alayer of work space degrees of freedom or mechanical linkages. Forexample, each of the work space interface elements 970 may correspond toa particular linkage of a manipulator assembly. The work spacecontroller 862 may be operable to perform calculations on outputs fromthe joint space controller 858 in one or more coordinate systems, suchas a Cartesian coordinate system, a polar coordinate system, etc., inany suitable number of dimensions (e.g., one dimension, two dimensions,three dimensions, greater than three dimensions, etc.). The calculationsperformed by the work space controller may include, for example, forwardkinematics, inverse kinematics, and other cart-space and null-spacealgorithms.

The joint space work space mapping 980 may be different for differentmanipulator assemblies, similar to the mappings discussed with referenceto FIGS. 9A and 9B. The joint space work space mapping 980 operates tomap joint space interface elements 960 to work space interface elements970. For example, the joint space work space mapping 980 may map outputsignals from the joint space controller 858 to the appropriate workspace interface elements 970, and similarly map output signals from thework space interface elements 970 to the appropriate joint spaceinterface elements 960. In some embodiments, the joint space work spacemapping 980 may be generated by, for example, the joint space work spacemapping unit 860.

In some embodiments, a joint space interface element may not be mappedat all to a work space interface element, resulting in a fixed frametransformation. For example, the first joint space interface element 960a is not mapped to any work space interface elements 970, and the firstwork space interface element 970 a is not mapped to any joint spaceinterface elements 960. This may be the case where the attachedmanipulator assembly does not have a degree of freedom corresponding tothe degree of freedom of the first joint space interface element 960 a(e.g., a secondary outer yaw joint).

In at least one embodiment, a single joint space interface element maybe mapped to a single work space interface element. For example, thesecond joint space interface element 960 b may be mapped to the secondwork space interface element 970 b. This may be the case where theattached manipulator assembly has a degree of freedom corresponding tothe degree of freedom of the second joint space interface element 960 b(e.g., an outer yaw degree of freedom), and that degree of freedomcorresponds to movement of a single link.

In at least one embodiment, a single joint space interface element maybe mapped to a number of work space interface elements. For example, thethird joint space interface element 960 c may be mapped to the thirdwork space interface element 970 c, the fourth work space interfaceelement 970 d, and the fifth work space interface element 970 e. Thismay be the case where a portion of the manipulator is a parallelmechanism. A parallel mechanism is where multiple physical joints movesimultaneously based on a single independent degree of freedom. One ofthe physical joints may be independently controlled, whereas motion ofthe other physical joints is dependent on movement of the independentlycontrolled joint. Examples of parallel mechanisms include snake tools(which move a tool tip around a curve instead of pivoting along a singleaxis), a parallelogram axis, etc.

FIG. 10 illustrates mappings between connector input/output elements,joint space interface elements, and work space interface elementsaccording to an embodiment. The various devices, interface elements,signals, and mappings are similar to those discussed with reference toFIGS. 9A and 9C, but in this case are shown as a single interconnectedsystem for mapping signals communicated to and from a manipulatorassembly to signals eventually processed by a work space controller.

In at least one embodiment, in addition to processing actual signalsreceived and communicated for controlling degrees of freedom of amanipulator assembly, the system may also be operable to performprocessing for simulated or phantom degrees of freedom. For example, insome embodiments, a manipulator assembly may be missing one or moredegrees of freedom, e.g., it may not be operable to roll. However, thecalculations executed by controllers, such as the joint space controller858 and/or work space controller 862, may be executed on the assumptionthat such degree(s) of freedom actually exist on the manipulatorassembly.

For example, the joint space controller 858, although it does notreceive input signals at the joint space interface elements 930 g and930 h, which may correspond to degrees of freedom such as a particularroll and a pitch of the manipulator assembly, nor does the joint spacecontroller 858 provide outputs at the joint space interface elements 930g and 930 h, the joint space controller 858 may nonetheless executealgorithms on the presumption that the degrees of freedom correspondingto the joint space interface elements 930 g and 930 h are beingcontrolled.

Accordingly, in some embodiments, the joint space work space mapping 980may include a mapping for processing signals associated with thesephantom degrees of freedom. For example, the joint space work spacemapping 980 may map the joint space interface element 930 gcorresponding to, e.g., a phantom pitch, to a work space interfaceelement 970 i, and may map the joint space interface element 930 hcorresponding to, e.g., a phantom yaw, to a work space interface element970 j. The work space controller 862 may then also perform calculations,similar to the joint space controller 858, on the presumption that thelinks corresponding to the work space interface elements 970 i and 970 jexist and are being controlled, although in reality they are not beingcontrolled and may not even exist on the connected manipulator assembly.

The number of connector input elements 910, connector output elements920, joint space interface elements 930, and work space interfaceelements 970 may be any suitable number for controlling a plurality ofmanipulator assemblies. In one embodiment, the number of joint spaceinterface elements is greater than the number of connector (input and/oroutput) elements. Such an arrangement may advantageously allow for anumber of different manipulator assemblies having a number of differentdegrees of freedom that may be controlled to all be controlled by thesame joint space controller. In cases where the number of joint spaceinterface elements is greater than the number of connector elements, theconnector elements may be mapped to a subset of the joint spaceinterface elements. Further, in cases where different manipulatorassemblies have the same number of degrees of freedom, but at least onedifferent degree of freedom (e.g., one manipulator includes an outerpitch degree of freedom, whereas another manipulator does not include anouter pitch degree of freedom but includes an additional tool degree offreedom), the connector elements for each of the manipulators may bemapped to different subsets of the joint space interface elements. Inother embodiments, the number of connector elements used by differentmanipulator assemblies may also be different.

In some embodiments, the number of work space interface elements isgreater than the number of joint space interface elements. Such anarrangement may advantageously allow for one joint space interfaceelement to be mapped to more than one cart space interface element,thereby facilitating control of a variety of mechanical linkages for agiven manipulator assembly degree of freedom.

Turning briefly to FIGS. 11A and 11B, FIG. 11A shows a first set ofgroupings 1100 of the degrees of freedom of a first manipulator assemblybeing controlled by the joint space interface elements 930, and FIG. 11Bshows a second set of groupings 1150 of the degrees of freedom of asecond manipulator assembly being controlled by the joint spaceinterface elements 930. Here, the second manipulator assembly isdifferent than the first manipulator assembly, and thus the groupingsmay also be different.

In some embodiments, algorithms executed by, e.g., the joint spacecontroller, may operate on only a subset of available inputs. Forexample, algorithms executed by the joint space controller 858 mayoperate on (e.g., acquire input from, process, and send outputs to) onlya subset of interface elements 930. The different algorithms, whilereceiving and operating on different sets of inputs, may be executedsimultaneously with one another. In situations where differentmanipulators (and, in some embodiments, different tools) aresequentially connected to the same connector of a support structure(i.e., one manipulator is swapped out for another manipulator), thejoints of the manipulator being controlled by the interface elements 930may change. As a result, the specific interface elements 930 whichalgorithms operate on may similarly change.

To account for these changes, different joint groupings may be used,where a joint group definition defines a particular joint grouping. Forexample, with reference to FIGS. 11A and 11B, which each illustrate aparticular interface grouping, interface elements 930 b through 930 emay correspond to outer joints of a first manipulator (FIG. 11A),whereas interface elements 930 a through 930 e may correspond to outerjoints of a second manipulator (FIG. 11B). When one or more algorithmsare structured to operate only on outer joints of a manipulator, whenthe first manipulator (FIG. 11A) is connected the algorithm(s) may readthe joint groupings definition for that manipulator and determine thatthe outer joints are fully defined using interface elements 930 bthrough 930 e. Thus, the algorithm(s) may operate only on thoseinterface elements. For example, a first algorithm may operate using the“outer” group of interface elements 930 b through 930 e, and a secondalgorithm may (simultaneously or sequentially) operate using the “tool”group of interface elements 930 f through 930 i. In contrast, when thesecond manipulator (FIG. 11B) is connected the algorithm may read thejoint groupings definition for that manipulator and determine that theouter joints are fully defined using interface elements 930 a through930 e. Thus, the algorithm(s) may operate on those interface elements(i.e., inclusive of interface element 930 a). For example, a thirdalgorithm may operate using the “outer” group of interface elements 930a through 930 e, and a fourth algorithm may operate using the “tool”group of interface elements 930 f through 930 i. Each of the first,second, third and fourth algorithms may be different from one anotherand optimized to execute on the particular group of interface elements.For example, one algorithm may be optimized to execute on a group ofinput elements associated with outer joints of a manipulator assembly,whereas another algorithm may be optimized to execute on a group ofinput elements associated with degrees of freedom of a tool.

A joint groupings definition may define all of the joint groupings usedby the algorithms in the controller for a particular manipulator. Forexample, with reference to FIG. 11A, the joint groupings definition maydefine the groupings for “joints”, “tool”, “jaw”, etc., for a firstmanipulator as depicted in FIG. 11A. With reference to FIG. 11B, adifferent joint groupings definition may define (in some cases,different) groupings for “joints”, “tool”, “jaw”, etc., for a secondmanipulator as depicted in FIG. 11B. The joint groupings definitions fordifferent manipulators may be stored at the controllers (e.g., insupport structure 850), or may be acquired from a suitable source (e.g.,from the manipulator, from a remote storage device, etc.) when or beforea manipulator is connected for use.

In the particular embodiments depicted in FIGS. 11A and 11B, at leastsome of the groupings illustrated in FIG. 11A are the same as thoseillustrated in FIG. 11B since the manipulator assembly being controlledby the interface elements of FIG. 11A has at least some of the samedegrees of freedom as those of the manipulator assembly being controlledby the interface elements of FIG. 11B. For example, both sets ofgroupings group the eighth joint space interface element 930 h and theninth joint space interface element 930 i into a “jaw” groupingindicating the joint space interface elements that operate to control ajaw of a tool. Further, both sets of groupings group the sixth jointspace interface element 930 f, seventh joint space interface element 930g, and eighth joint space interface element 930 h, into a “wrist”grouping indicating the joint space interface elements that operate to acontrol a wrist of the tool.

Further, at least some of the groupings illustrated in FIG. 11A aredifferent than those illustrated in FIG. 11B since the manipulatorassembly being controlled by the interface elements of FIG. 11A does notinclude a secondary outer yaw joint, whereas the manipulator assemblybeing controlled by the interface elements of FIG. 11B does include asecondary outer yaw joint. Accordingly, a grouping defined as “outer”,which identifies the joint space interface elements that operate tocontrol the manipulator arm, is defined for the first set of groupings1100 to include the second, third, fourth, and fifth joint spaceinterface elements 930 b to 930 e. In contrast, the grouping defined as“outer” is defined for the second set of groupings 1100 to include thefirst, second, third, fourth, and fifth joint space interface elements930 a to 930 e, as the secondary outer yaw joint corresponds to a degreeof freedom of the manipulator arm rather than a connected tool.

As mentioned, algorithms executed by the joint space controller mayselect the signals to use from the joint space interface elements 930based on input data received for the algorithm(s). For example, inputdata from a sensor, actuator, and/or work space controller may indicateto the algorithm(s) which joint space interface elements correspond to,e.g., tool degrees of freedom, and thus the joint space controller mayuse signals received from (and communicate processed signals to) theappropriate joint space interface elements. In other embodiments, thesignals may be selected based on a predetermined correspondence betweenthe joint space interface elements and degrees of freedom of themanipulator assembly. For example, upon connecting a particularmanipulator assembly, grouping information indicating the groupings ofdifferent joint space interface elements may be loaded that correspondsto the particular manipulator assembly. Thus, the joint space controllermay use signals received from (and communicate processed signals to) theappropriate joint space interface elements defined by the particulargrouping.

Turning now to FIGS. 12A and 12B, FIG. 12A shows a connector/joint spacemap 1200 according to an embodiment, while FIG. 12B shows a jointspace/work space map 1250 according to an embodiment. The maps may bestored in any suitable element of the robotic system, such as a storageelement in the support structure 850. The connector/joint space map 1200is a map for mapping (e.g., first mapping 940 and/or second mapping 950)connector elements 1210 (e.g., contacts 854) to joint space interfaceelements 1220 (e.g., joint space interface elements 930). Theconnector/joint space map 1200 may include any suitable number ofconnector elements, such as A1 to Ai (where i is any integer), and anysuitable number of joint space interface elements, such as J1 to Jk(here k is any integer). A mapping indicator 1230 may indicate aparticular mapping between a particular connector element and aparticular joint space interface element. For example, with reference toFIG. 9A, connector element “A2” may correspond to second connector inputelement 910 b, and the joint space interface element “J2” may correspondto the second joint space interface element 960 b. Accordingly, themapping indicator 1230 may be operable to map a signal between thesecond connector input element 910 b and the second joint spaceinterface element 960 b.

The joint space/work space map 1250 is a map for mapping (e.g., jointspace work space mapping 980) joint space interface elements 1260 (e.g.,joint space interface elements 930) to work space interface elements1220 (e.g., work space interface elements 970). The joint space/workspace map 1250 may include any suitable number of joint space elements,such as J1 to Jk (where k is any integer), and any suitable number ofwork space interface elements, such as W1 to Wm (where m is anyinteger). A mapping indicator 1280 may indicate a particular mappingbetween a particular joint space element and a particular work spaceinterface element. For example, with reference to FIG. 9C, joint spaceinterface element “J2” may correspond to the second joint spaceinterface element 960 b, and the work space interface element “W1” maycorrespond to the second work space interface element 970 b.Accordingly, the mapping indicator 1280 may be operable to map a signalbetween the second joint space interface element 960 b and the secondwork space interface element 970 b.

Further, the maps may be operable to map one element to more than oneelement. For example, again with reference to FIG. 9C, joint spaceinterface element “J3” may correspond to the third joint space interfaceelement 960 c, and the work space interface elements “W2”, “W3”, and“W4” may respectively correspond to the second, third, and fourth workspace interface elements 970 c to 970 e. First, second, and thirdmapping indicators 1282, 1284, and 1286 may thus be operable to map asignal between the third joint space interface element 960 c and each ofthe second, third, and fourth work space interface elements 970 c to 970e.

The support structure in certain embodiments includes various interfaceelements and controllers such as connector elements, interface elements,mappings, joint space controllers, work space controllers, etc. However,it will be appreciated by those of ordinary skill in the art that theinput device could operate equally well by having fewer or a greaternumber of components than are illustrated in FIGS. 9A to 12B. It will befurther appreciated by those of ordinary skill in the art that theelements described with reference to each of FIGS. 9A to 12B need notrequire the particular mappings nor the particular degrees of freedomillustrated and discussed with reference to these embodiments. Forexample, some of the degrees of freedom may correspond to movement of aconnected manipulator arm, whereas other degrees of freedom maycorrespond to movement of a connected tool, whereas yet other degrees offreedom may correspond to actuation of a tool or manipulator arm. Thus,the depiction of the various elements in FIGS. 9A to 12B should be takenas being illustrative in nature, and not limiting to the scope of thedisclosure.

FIG. 13A shows a patient side cart 1300 similar to patient side cart 22depicted in and discussed with reference to FIG. 4. However, in thisembodiment, the manipulator arms are individually identified and includea first manipulator arm 100A, a second manipulator arm 100B, a thirdmanipulator arm 100C, and a fourth manipulator arm 100D. The firstmanipulator arm 100A is coupled to an imaging device 28, whereas thesecond, third and fourth manipulator arms (100B, 100C, 100D) are eachcoupled to a respective surgical tool 26.

FIG. 13B shows a patient side cart 1350 similar to patient side cart1300. However, in this embodiment, the first manipulator arm 100A iscoupled to a surgical tool 26 whereas the second manipulator arm 100B iscoupled to the imaging device 28. That is, the imaging device has beenswapped with one of the surgical tools.

In many embodiments, the imaging device 28 can be swapped with asurgical tool 26 at any suitable time, including during a medicalprocedure on a patient without turning off or otherwise rebooting thesoftware executing on the patient side cart 1350 or other elements ofthe MIRS system 10 (FIG. 1). Swapping the imaging device 28 with asurgical tool 26 may facilitate changes in the field of view of theimaging device 28, and may facilitate changes in the accessible workspace available to the swapped out surgical tool.

In at least one embodiment, a frame of reference of the surgical tooltipis changed as a result of moving the imaging device 28 (e.g., a camera)from one manipulator to another manipulator. For example, whenperforming surgery it is generally desirable to rearrange things inspace so that the frame of reference of a surgical tooltip relative tothe camera tip matches the frame of reference of an input device used bya surgeon to control the surgical tool relative to an image displayviewed by the surgeon. When a surgical tool is coupled to a firstmanipulator arm, and an imaging device is coupled to a secondmanipulator arm, the surgeon (via, e.g., an input device) is driving thesurgical tool via the first manipulator. The surgical tool tip framerelative to the imaging device frame is thus used for tele-operation.However, the surgical tool and imaging device are swapped with oneanother, the surgeon ends up driving the imaging device. As a result,the system (e.g., one or more algorithms executed by the joint spacecontroller and/or the work space controller) defines an arbitrary frameof reference (i.e., a world frame) such that the imaging device framerelative to the world frame is used for tele-operation.

One skilled in the art would recognize that various techniques disclosedherein can be used to assist in facilitating the swapping of imagedevices and surgical tools. For example, the first manipulator arm 100Aand the second manipulator arm 100B may each be similar to manipulator820 (FIGS. 8A to 8C), and each of the imaging device 28 and surgicaltool 26 may be similar to the tool 840. Accordingly, each of the firstmanipulator arm 100A and the second manipulator arm 100B may include aconnector (e.g., connector 834) that is shaped to engage a correspondingconnector of the imaging device 28 and the surgical tool 26.

Various processing that may be performed by, e.g., support structure 850to facilitate the hot-swapping of imaging devices and surgical tools isfurther described with reference to FIGS. 14 to 16B. One skilled in theart would recognize that embodiments are not limited to swapping imagingdevices with surgical tools, but include swapping surgical tools withother surgical tools, imaging devices with other types of imagingdevices, etc.

FIG. 14 shows a sequence of operations that may be used to control onefrom a number of possible tools connected to the same manipulator arm.Or, more generally, to control a number of different manipulatorassemblies, where a manipulator assembly includes both a manipulator armand a tool. The operations may be performed by any suitable controller,such as any of those described herein with reference to supportstructure 850 (FIG. 8A to 8C).

In operation 1410, it is determined whether a tool is connected to amanipulator. For example, joint space controller 858 (or a differentcontroller) may determine whether tool 840 is connected to manipulatorarm 820. When it is determined that a tool is not connected to themanipulator arm, the system may continue to monitor the manipulator armto determine whether a tool is connected. Otherwise, processing maycontinue to operation 1420.

In operation 1420, a mapping for the connected tool is acquired. Themapping may include one or more mappings, such as those described withreference to FIGS. 8A through 12B. For example, the mappings may includea connector/joint space map 1200 (FIG. 12A), a joint space/work spacemap 1250 (FIG. 12B), or any other suitable map that associated with theconnected tool. The mappings may be acquired using any one or more of anumber of techniques, some of which are described with reference to FIG.15.

Once mappings for the tool are acquired, processing continues tooperation 1430. In operation 1430, the connected tool is controlledusing the acquired mapping. For example, the tool may be controlledusing the joint space controller 858 and the work space controller 862using the connector/joint space map and the a joint space/work spacemap. Various techniques for using one or more maps to control a tool aredescribed with reference to FIGS. 8A through 12B. Any one or more ofthose techniques are applicable for controlling the connected tool.Further techniques for controlling the connected tool are described withreference to FIGS. 16A and 16B.

In operation 1440 it is determined whether the tool is removed from themanipulator. For example, a controller determines whether the tool 840is removed from the manipulator arm 820. If it is determined that a toolis not removed from the manipulator arm, then tool may continue to becontrolled using the acquired mappings. Otherwise, processing continuesto operation 1450.

In operation 1450, it is determined whether a new tool is connected tothe manipulator. This operation is similar to operation 1410. If it isdetermined that a new tool is connected, then a mapping for the new toolmay be acquired in operation 1420, and the new tool may then becontrolled using the new mapping in operation 1430. Otherwise, thesystem may continue to monitor for connection of a new tool.

As a result of the mappings, the same software kernel can be used byprocessors (e.g., the joint space controller 858 and/or the work spacecontroller 862) that control the different tools connected to the samemanipulator arm. The software kernel may include the functionality ofall different types of tools that can be connected to the manipulatorarm, such as acquiring images from an imaging device, actuating forceps,etc. The software kernel then has the capability to actuate any functionof all of the tools, while the tool mappings provide the appropriatesignal routing such that different tools having different degrees offreedom can effectively be controlled when connected to the samemanipulator arm.

Referring now to FIG. 14A, an example of a sequence of operations 1421for acquiring mappings for one or a tool may be used when, for example,a tool mounted to a manipulator of the patient side cart may be an imagecapture device or a surgical instrument, so as to facilitate changes toa frame of reference when appropriate. This exemplary sequence ofoperations may be included within operation 1420 of FIG. 14 whenacquiring mapping, and may help maintain correlation between movementcommands input by a system user (such as by moving a handle of an inputdevice) and corresponding movement of a tip of a surgical instrument asimaged by an image capture device (including an image capture devicenewly mounted to a manipulator) and as displayed to the system user.Additional details regarding the correlation between the input anddisplay of the moving instrument can be understood, for example, withreference to U.S. Pat. No. 6,424,885, entitled “Camera ReferencedControl in a Minimally Invasive Surgical Apparatus,” the full disclosureof which is incorporated herein by reference.

In operation 1423, it is determined whether a camera or other imagecapture device has been connected to a manipulator. The mounting of acamera to a manipulator may be determined by signals transmitted betweenthe camera and the manipulator, or by an input from a system user. Inoperation 1425, it is determined that the camera or other image capturedevice that has been mounted to the manipulator is the reference camera.Where only one camera is to be mounted to the manipulator system and/orwhere the camera that has been mounted is the first camera to be mountedto the manipulator system the system may, in response, designate thecamera that has been mounted as the reference camera. Optionally, thedesignation of the reference camera may be in response to an input by asystem user, a type of camera, or the like. If no camera has beenmounted or the mounted camera is not the reference camera the system maycontinue to monitor for mounting of the reference camera.

In operation 1427, when a reference camera has been mounted to themanipulator system the system acquires a mapping of a camera mapping.For example, referring to FIG. 14B before a camera swap operation acamera 28 may have a camera reference coordinate frame of referenceR_(cam) that can be identified (optionally relative to a worldcoordinate reference system) by the system controller using signals fromthe potentiometers or other joint state sensors associated with each ofthe joints of the robotic arm (including the first manipulator 100A asshown in FIG. 13A) and set-up joints or other structures supporting thecamera), together with known attributes of the robotic arms (such as thekinematics of the system associated with the joint state sensors)Similar mappings of each of the tool tips in the field of view of thecamera of surgical instruments 26 a, 26 b, 26 c and their associatedmanipulators 100B, 100C, 100D to R_(cam) can be used by the controllerto help maintain correlation between movement command vectors and theresulting tool tip motions. Referring to FIG. 14C, after a tool swapoperation, camera 28 may be mounted to manipulator 100B or another ofthe manipulators. Operation 1427 acquires a new mapping of the newlymounted camera 28 using the joint state signals of manipulator 100B toderive a new camera reference frame R_(cam)′. The new camera referenceframe may then be used to determine joint commands for movement of thecamera and/or for movement of all the surgical instruments mounted tothe other manipulators, including surgical instrument 26 d mounted tomanipulator 100A (which previously supported camera 28). Note thatinstrument 26 d may optionally be the same instrument 26 b which wasremoved from manipulator 100B prior to the tool swap, or may be adifferent instrument. Similarly, camera 28 may be the same camera thatwas supported by manipulator 100A prior to the tool swap or may be adifferent camera.

In operation 1429, default master-slave associations between inputdevices and surgical instruments may be set. The associates may bedetermined, for example, in response to relative positions of the inputdevices relative to the user display and tool tips of the surgicalinstruments relative to the camera reference frame. Hence, right andleft input devices are associated with surgical instruments which appearin the display to be to the right and left side of the workspace,respectively. Note that the user may manually set associates whendesired, and that when the arrangement of tools is not amenable toautomated or default associations (such as in the arrangement of FIG.13B where left and right tools are somewhat arbitrary) the system mayprompt and/or wait for user input on appropriate master-slaveassociations. Master-Slave associates in more complex systems havingmultiple concurrent users may allow user control over those associatesas described in more detail in U.S. Pat. No. 8,666,544 entitled“Cooperative Minimally Invasive Telesurgical System,” the disclosure ofwhich is also incorporated herein by reference.

In operation 1431, the master input devices may be moved (in orientationand/or position) to match the associated slave tool tips. Movement ofthe masters may be performed by actuating motors of the masters asdescribed, for example, in U.S. Pat. No. 6,364,888 entitled “Alignmentof Master and Slave in a Minimally Invasive Surgical Apparatus,” thefull disclosure of which is incorporated herein by reference. Once themasters have been moved the system may be ready to initiate telepresencefollowing, or for another desired operations in preparation forfollowing.

Turning now to FIG. 15, FIG. 15 shows a sequence of operations foracquiring mappings for a tool according to an embodiment. In operation1422, it is determined whether the tool has the mappings stored therein.For example, the mappings may be stored on a storage medium of the tool.

When it is determined that the tool does not have the mappings storedtherein, the processing may continue to operation 1428. In operation1428, the mappings are received from a source other than the tool. Forexample, the hardware mapping unit 856 and/or the joint space/work spacemapping unit 860 may receive the mappings from a source other than thetool. The source may be any electronic device other than the tool thathas mappings stored thereon. For example, the source may be a remoteserver, a local hard drive, etc. The mappings received from such asource may then subsequently be used for controlling the tool.

When it is determined that the tool does have the mappings storedtherein, the processing may continue to operation 1424. In operation1424, the mappings are received from the tool. For example, the hardwaremapping unit 856 and/or the joint space/work space mapping unit 860 mayreceive the mappings from storage element of the tool. This may be wiredor wireless communication using any suitable communication protocol.Once the mappings are received from the tool, then processing maycontinue to operation 1426.

In operation 1426 it is determined whether the mappings received fromthe tool are valid. For example, a controller such as the hardwaremapping unit 856, work space controller 862, or other suitablecontroller may determine whether the received mappings are valid. Thismay include determining whether the mappings are out of date, arecorrupt, are for the wrong tool, etc. If it is determined that themappings are invalid, then processing may continue to operation 1428 aspreviously described. Otherwise, the mappings received from the tool maysubsequently be used to control the tool.

It should be recognized that techniques for acquiring the mappings for atool are not limited to those described with reference to FIG. 15.Rather, embodiments also include other techniques for acquiringmappings. For example, the controllers may simply download and usemappings provided by the tool or a source other than the tool. Foranother example, the controllers may have locally stored mappings foreach tool. One skilled in the art would recognize other variations, andsuch variations are intended to be covered within the scope of thisapplication.

Turning now to FIGS. 16A and 16B, FIG. 16A shows a sequence ofoperations 1430 that may be used to control a tool using an acquiredmapping in accordance with one embodiment. In operation 1602, aplurality of sensor signals are received at a plurality of joint spaceinterface elements. For example, with reference to FIG. 9A, sensorsignals 912, 914, etc. may be received at joint space interface elements930 from the connector input elements 910 via the first mapping 940.

In operation 1604, the received sensor signals may be processed with ajoint space controller. For example, the sensor signals may be processedby joint space controller 858. In one embodiment, the joint spacecontroller may execute algorithms on the received signals in joint spaceand then provide the output signals to control the connected manipulatorassembly. In another embodiment, such as that discussed with referenceto FIG. 16B, the joint space controller may execute algorithms on thereceived signals in joint space and then provide the output signals toanother controller, such as a work space controller 862, for furtherprocessing.

In some embodiments, at least one additional signal may be processed inaddition to the received sensor signals. For example, as discussed withreference to FIG. 10, the system may be operable to perform processingsimulated or phantom degrees of freedom. Accordingly, the joint spacecontroller 858 may be operable to process an additional signal such as aphantom input at the joint space interface element 930 g and/or thejoint space interface element 930 h.

In operation 1606, the processed signals are output to actuators via thejoint space interface elements. For example, the processed signals maybe output from the joint space interface elements 930 and sent to theconnector output elements 920 via the second mapping 950, where theprocessed signals operate to control one or more degrees of freedom ofthe connected manipulator assembly.

In embodiments where the manipulator assembly is changed (e.g., swappingof a manipulator arm and/or swapping of a tool), the same operations maysubsequently be performed for the new manipulator assembly. For example,if an imaging device is first connected to a manipulator arm, aplurality of sensor signals may be received via an acquired mappingunique to that imaging device. If the imaging device is then swappedwith a surgical tool, a new plurality of sensor signals are received atthe same joint space interface elements via an acquired mapping uniqueto the surgical tool. In such a fashion, different manipulatorassemblies can be controlled using a single software kernel.

Turning now to FIG. 16B, FIG. 16B shows a sequence of operations 1430that may be used to control a tool using an acquired mapping inaccordance with another embodiment. In one embodiment, these operationsmay be performed as operations in step 1430 of FIG. 14. In anotherembodiment, these operations may be performed as part of operation 1604(FIG. 16A).

In operation 1652, joint layer output signals are received at aplurality of work space interface elements via the acquired mapping. Forexample, with reference to FIG. 9C, signals output from the joint spacecontroller 858 may be received at work space interface elements 970 fromthe joint space interface elements 960 via the joint space work spacemapping 980. Such joint layer output signals may correspond to thoseprocessed signals received from a first manipulator assembly, and thusmay correspond to degrees of freedom of the first manipulator assembly.

In one embodiment, a single work space interface element (e.g., 970 b)may receive an output signal from a single corresponding joint layerinterface element (e.g., 960 b), where in other embodiments, a number ofwork space interface elements (e.g., 970 c, 970 d, and 970 e) mayreceive the same output signal from a single joint layer interfaceelement (e.g., 960 c). Further, in at least one embodiment, a work spaceinterface element (e.g., 970 i) may receive an output signal from ajoint space interface element corresponding to a simulated degree offreedom of the manipulator assembly (e.g., 930 g).

In operation 1654, the joint layer output signals are processed with awork space controller. For example, the output signals may be processedby work space controller 865. In one embodiment, the work spacecontroller may execute algorithms on the received signals in the workspace and then provide the output signals back to the joint spacecontroller to control the connected manipulator assembly. In anotherembodiment, the work space controller may communicate the processedsignals to other elements of the control system, such as the masterinput device.

In operation 1656, the processed signals are output to a plurality ofjoint space interface elements. For example, the processed signals maybe output from the work space interface elements 970 to the joint spaceinterface elements 960 via the joint space work space mapping 980, wherethe signals may be further processed by the joint space controller 858and, in some embodiments, subsequently used to control the firstmanipulator assembly.

It should be appreciated that the specific operations illustrated inFIGS. 14 to 16B provide particular methods of controlling manipulatorassemblies according to certain embodiments of the present invention.Other sequences of operations may also be performed according toalternative embodiments. For example, alternative embodiments of thepresent invention may perform the operations outlined above in adifferent order. Moreover, the individual operations illustrated inFIGS. 14 to 16B may include multiple sub-operations that may beperformed in various sequences as appropriate to the individualoperation. Furthermore, additional operations may be added or existingoperations removed depending on the particular applications. One ofordinary skill in the art would recognize and appreciate manyvariations, modifications, and alternatives.

Further, it should be recognized that while the terms tools,instruments, surgical tools, surgical instruments and the like are oftenused interchangeably, in some embodiments they may not have identicalmeanings. For example, surgical instruments and surgical tools may referto instruments or tools that are used for actively manipulating apatient, such as forceps, clamps, cutters, suction tubes, needles,drills, etc. In contrast, non-surgical instruments or tools may refer tothose that are not used for actively manipulating a patient, such as animaging device. The general term of tools or instruments may broadlycover both surgical and non-surgical instruments or tools.

The operations described in this application may be implemented assoftware code to be executed by one or more processors using anysuitable computer language such as, for example, Java, C++ or Perlusing, for example, conventional or object-oriented techniques. Thesoftware code may be stored as a series of instructions, or commands ona computer-readable medium, such as a random access memory (RAM), aread-only memory (ROM), a magnetic medium such as a hard-drive or afloppy disk, or an optical medium such as a CD-ROM. Any suchcomputer-readable medium may also reside on or within a singlecomputational apparatus, and may be present on or within differentcomputational apparatuses within a system or network.

The present invention can be implemented in the form of control logic insoftware or hardware or a combination of both. The control logic may bestored in an information storage medium as a plurality of instructionsadapted to direct an information processing device to perform a set ofsteps disclosed in embodiments of the present invention. Based on thedisclosure and teachings provided herein, a person of ordinary skill inthe art will appreciate other ways and/or methods to implement thepresent invention.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing embodiments (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted. The term “connected” is to beconstrued as partly or wholly contained within, attached to, or joinedtogether, even if there is something intervening. Recitation of rangesof values herein are merely intended to serve as a shorthand method ofreferring individually to each separate value falling within the range,unless otherwise indicated herein, and each separate value isincorporated into the specification as if it were individually recitedherein. All methods described herein can be performed in any suitableorder unless otherwise indicated herein or otherwise clearlycontradicted by context. The use of any and all examples, or exemplarylanguage (e.g., “such as”) provided herein, is intended merely to betterilluminate embodiments and does not pose a limitation on the scopeunless otherwise claimed. No language in the specification should beconstrued as indicating any non-claimed element as essential to thepractice of at least one embodiment.

Preferred embodiments are described herein, including the best modeknown to the inventors. Variations of those preferred embodiments maybecome apparent to those of ordinary skill in the art upon reading theforegoing description. The inventors expect skilled artisans to employsuch variations as appropriate, and the inventors intend for embodimentsto be constructed otherwise than as specifically described herein.Accordingly, suitable embodiments include all modifications andequivalents of the subject matter recited in the claims appended heretoas permitted by applicable law. Moreover, any combination of theabove-described elements in all possible variations thereof iscontemplated as being incorporated into some suitable embodiment unlessotherwise indicated herein or otherwise clearly contradicted by context.The scope of the invention should, therefore, be determined not withreference to the above description, but instead should be determinedwith reference to the pending claims along with their full scope orequivalents.

What is claimed is:
 1. A system comprising: a plurality of manipulators;and a controller configured to: detect mounting of an imaging device toa first manipulator of the plurality of manipulators; determine a firstreference frame for the imaging device based on the mounting of theimaging device to the first manipulator; control a tool relative to thefirst reference frame by controlling a relative position and orientationof a tip of the tool relative to the imaging device in the firstreference frame by correlating movement of a master input control tomovement of the tool in the first reference frame; detect mounting ofthe imaging device to a second manipulator of the plurality ofmanipulators, the second manipulator being different from the firstmanipulator; determine a second reference frame for the imaging devicebased on the mounting of the imaging device to the second manipulator;and control the tool relative to the second reference frame.
 2. Thesystem of claim 1, wherein the second reference frame is a worldreference frame.
 3. The system of claim 1, wherein the first referenceframe is determined based on kinematics of the first manipulator.
 4. Thesystem of claim 1, wherein to control the tool relative to the firstreference frame, the controller is configured to teleoperate the toolbased on commands received from the master input control.
 5. The systemof claim 1, wherein to control the relative position and orientation ofthe tip of the tool relative to the imaging device in the firstreference frame, the controller is configured to control the tool basedon a relative position and orientation of the master input controlrelative to a user display.
 6. The system of claim 1, wherein to controlthe tool relative to the second reference frame, the controller isconfigured to control the relative position and orientation of the tipof the tool relative to the imaging device in the second reference framebased on a relative position and orientation of the master input controlrelative to a user display.
 7. The system of claim 1, wherein inresponse to determining the second reference frame for the imagingdevice, the controller is configured to move the master input control sothat the relative position and orientation of the master input controlrelative to a user display corresponds to a relative position andorientation of the tip of the tool relative to the imaging device in thesecond reference frame.
 8. The system of claim 1, wherein: the tool ismounted to a third manipulator of the plurality of manipulators; andcontrol of the tool is further based on kinematics of the thirdmanipulator.
 9. The system of claim 1, wherein: while the imaging deviceis mounted to the first manipulator and the tool is mounted to thesecond manipulator, control of the tool is further based on kinematicsof the second manipulator; and while the imaging device is mounted tothe second manipulator and the tool is mounted to the first manipulator,control of the tool is further based on kinematics of the firstmanipulator.
 10. A method for controlling a system, the methodcomprising: detecting, by a controller of the system, mounting of animaging device to a first manipulator of the system; determining, by thecontroller, a first reference frame for the imaging device based on themounting of the imaging device to the first manipulator; controlling, bythe controller, a tool relative to the first reference frame bycontrolling a relative position and orientation of a tip of the toolrelative to the imaging device in the first reference frame bycorrelating movement of a master input control to movement of the toolin the first reference frame; detecting, by the controller, mounting ofthe imaging device to a second manipulator of the system, the secondmanipulator being different from the first manipulator; determining, bythe controller, a second reference frame for the imaging device based onthe mounting of the imaging device to the second manipulator; andcontrolling, by the controller, the tool relative to the secondreference frame.
 11. The method of claim 10, wherein determining thefirst reference frame comprises determining the first reference framefurther based on kinematics of the first manipulator.
 12. The method ofclaim 10, wherein controlling the tool relative to the first referenceframe comprises controlling the tool based on a relative position andorientation of the master input control relative to a user display. 13.The method of claim 10, wherein controlling the tool relative to thesecond reference frame comprises controlling the relative position andorientation of the tip of the tool relative to the imaging device in thesecond reference frame by correlating movement of the master inputcontrol to movement of the tool in the second reference frame.
 14. Themethod of claim 10, wherein controlling the tool relative to the secondreference frame comprises controlling the tool based on a relativeposition and orientation of the master input control relative to a userdisplay.
 15. The method of claim 10, further comprising: in response todetermining the second reference frame for the imaging device, movingthe master input control so that the relative position and orientationof the master input control relative to a user display corresponds to arelative position and orientation of the tip of the tool in the secondreference frame for the imaging device.
 16. The method of claim 10,wherein: while the imaging device is mounted to the first manipulatorand the tool is mounted to the second manipulator, controlling of thetool is further based on kinematics of the second manipulator; and whilethe imaging device is mounted to the second manipulator and the tool ismounted to the first manipulator, controlling of the tool is furtherbased on kinematics of the first manipulator.
 17. A non-transitorycomputer readable medium comprising a plurality of instructions whichwhen executed by one or more processors associated with a system areadapted to cause the one or more processors to perform a methodcomprising: detecting mounting of an imaging device to a firstmanipulator of the system; determining a first reference frame for theimaging device based on the mounting of the imaging device to the firstmanipulator; controlling a tool relative to the first reference frame bycontrolling a relative position and orientation of a tip of the toolrelative to the imaging device in the first reference frame bycorrelating movement of a master input control to movement of the toolin the first reference frame; detecting mounting of the imaging deviceto a second manipulator of the system, the second manipulator beingdifferent from the first manipulator; determining a second referenceframe for the imaging device based on the mounting of the imaging deviceto the second manipulator; and controlling the tool relative to thesecond reference frame.
 18. The non-transitory computer readable mediumof claim 17, wherein the method further comprises: while the imagingdevice is mounted to the first manipulator and the tool is mounted tothe second manipulator, controlling of the tool is further based onkinematics of the second manipulator; and while the imaging device ismounted to the second manipulator and the tool is mounted to the firstmanipulator, controlling of the tool is further based on kinematics ofthe first manipulator.
 19. The non-transitory computer readable mediumof claim 17, wherein controlling the tool relative to the firstreference frame comprises controlling the tool based on a relativeposition and orientation of the master input control relative to a userdisplay.
 20. The non-transitory computer readable medium of claim 17,wherein controlling the tool relative to the second reference framecomprises controlling the relative position and orientation of the tipof the tool relative to the imaging device in the second reference frameby correlating movement of the master input control to movement of thetool in the second reference frame.